top button
Flag Notify
    Connect to us
      Site Registration

Site Registration

Listing modules from all installed packages in python

0 votes
469 views

I'm trying to write a function that programmatically obtains and returns the exact location of all first-level modules for all installed packages.

For example, if the packages named 'django' and 'django-debug-toolbar' are installed, I'd like this function to return something like:

installed_modules()
/Users/my_user/.virtualenvs/my_venv/lib/python2.6/site-packages/django
/Users/my_user/.virtualenvs/my_venv/src/debug_toolbar

That is, this function needs to consider all installed packages, including those that have been installed in "edit" mode (i.e. in the src/ folder). Note also that the main module for the 'django-debug-toolbar' is in fact named 'debug_toolbar'.

So far the closest I've been to retrieving the list of first-level modules is as follows:

 import os
 import pkg_resources
 import setuptools

 pkgs = set()

 for dist in pkg_resources.working_set:
 if os.path.isdir(dist.location):
 for pkg in setuptools.find_packages(dist.location):
 if '.' not in pkg:
 pkgs.add(pkg)

The idea is then to loop through that list of modules, import them and get their exact locations by fetching their __file__ attribute values.

However, this feels very hackish and I don't think it's actually quite correct either. I'm sure there must be a better way. If possible I'd also like to avoid having to use setuptools.

Does anyone have any tips on how to achieve this?

posted Jun 9, 2013 by anonymous

Share this question
Facebook Share Button Twitter Share Button LinkedIn Share Button

2 Answers

0 votes

print 'n'.join([re.findall("from '(.*)'",str(v))[0] for k,v in sys.modules.items() if str(v).find('from')>-1])

answer Jun 9, 2013 by anonymous
0 votes

Just realized that you've asked for installed packages. Perhaps the following will do the trick. I don't know why the 'lib-tk' isn't included. Why not?

toplevel_packages = ['%s\%s'%(ml.path,name)for ml,name,ispkg in pkgutil.iter_modules() if ispkg]
print 'n'.join(toplevel_packages)

answer Jun 9, 2013 by anonymous
Similar Questions
+1 vote
Is there a way to install ALL Python packages?

I'd like to install ALL Python packages on my machine. Even if it takes up 4-5GB, or more, I'd like to get everything, and then use it when I need it. Now, I'd like to import packages, like numpy and pandas, but nothing will install. I figure, if I can just install everything, I can simply use it when I need it, and if I don't need it, then I just won't use it.

I know R offers this as an option. I figure Python must allow it too.

Any idea how to grab everything?

0 votes
How to view all installed python libraries in terminal ?

Is there any command available to view the installed libraries. Please suggest.

+2 votes
Python: Running code from source that includes extension modules

I have started looking into distutils because I need to write an extension module in C (for performance reasons) and distutils seems to be the most straight-forward way.

I have had success building a C file into a Python extension module using "python setup.py build" but I am wondering what the recommended way for using that module during development is. While writing Python code I am used to just run the code from the source directory. But the built extension modules .so of course does not just end up on sys.path magically.

So how do I run my code so it will find the built extension module? Do I pass the output directory on the command line manually or is there some other solution? I would like to still be able to run the code from the source directory as I am using PyCharm to edit and debug the code.

+1 vote
importing modules from Shogun to python

Attempting to import modules from Shogun to python from a non-standard python directory i.e. from my /home/xxx directory. Is there a way on ubuntu to selectively some modules, scripts, data from one directory and others modules, scripts from another directory. In other words, is there a file(s) that provide pointers to where these modules are located.

+2 votes
Does the order in which the modules are placed in a file matters in Python?

Iam on python 2.7 and linux .I need to know if we need to place the modules in a particular or it doesn't matter at all while writing the program.

For Example

import os
import shlex
import subprocess
import time
import sys
import logging
import plaftform.cluster
from util import run

def main():
 """ ---MAIN--- """

if __name__ == '__main__':
 main()

In the above example :

I am guessing may be the python modules like os , shlex etc come first and later the user defined modules like import plaftform.cluster etc

Sorry if my question sounds dump , I was running pep8 and don't see its bothered much about it

...