Why on CentOS, python consumes too much memory ?

Previously, we found that our python scripts consume too much memory. So I use python's resource module to restrict RLIMIT_AS's soft limit and hard limit to 200M.
On my RHEL5.3, it works OK. But on CentOS 6.2 + python2.6.6, it reports memory error(exceeding 200M). And I tested with a very small script, and result is out of my expect, it still use too much memory on my CentOS 6.2 python:
I could understand that 64 bit machines will occupy more virtual memory than that on 32 bit, because the length of some types are not the same. But I don't know why they differs so greatly(6M to 180M), Or is this only caused by that python2.6 on CentOS 6.2's memory allocation is different from python's default one? Could you kindly give me some clues?

1 Answer

I found that it was caused by not by python but by /usr/lib/locale/locale-archive, the same problem as that described in http://illiterat.livejournal.com/4615.html.

answer Jul 18, 2013 by anonymous

Similar Questions

+1 vote

How best to install path.py package in Python 3.4 environment on Centos 5?

I want to install path.py in my Python 3.4 environment on a Centos 5 box. My /usr/local/bin/ contains:

easy_install-3.4 
python3.4  
etc.

We are behind a proxy server and I tried this:

# /usr/local/bin/easy_install-3.4 path.py 

Searching for path.py 
Reading https://pypi.python.org/simple/path.py/ 
Download error on https://pypi.python.org/simple/path.py/: hostname '172.29.68.1  
' doesn't match either of 'www.python.org', 'python.org', 'pypi.python.org',  
'docs.python.org', 'testpypi.python.org', 'bugs.python.org', 'wiki.python.org',  
'hg.python.org', 'mail.python.org', 'packaging.python.org', 'pythonhosted.org',  
'www.pythonhosted.org', 'test.pythonhosted.org', 'us.pycon.org', 'id.python.org' --  
Some packages may not be found!

Couldn't find index page for 'path.py' (maybe misspelled?), Am I best to use pip or easy_install? also if easy_install, how can I fix the above error?

0 votes

Django 1.5 Upgrade on CentOS

On my Mac, running python 2.7, I upgraded from 1.4 to 1.5 by doing this:

rm -rf /Library/Python/2.7/site-packages/django

Followed by python setup.py install in the dir I untar-ed Django-1.5.1 to.This all worked fine.

On a Centos system running python 2.6 I did: rm -rf /usr/lib/python2.6/site-packages/django followed by the
setup.py install. My django app is working, and appears to be running 1.5, however anytime I use manage.py (for collectstatic or syncdb or test, for example) I get these messages:

/usr/lib/python2.6/site-packages/django/core/management/__init__.py:465:
DeprecationWarning: The 'execute_manager' function is deprecated, you likely need to update your 'manage.py'; please see the Django 1.4 release notes (https://docs.djangoproject.com/en/dev/releases/1.4/).

/usr/lib/python2.6/site-packages/django/core/management/__init__.py:409:
DeprecationWarning: The 'setup_environ' function is deprecated, you likely need to update your 'manage.py'; please see the Django 1.4 release notes (https://docs.djangoproject.com/en/dev/releases/1.4/).

So it seems I still have some 1.4 stuff around. I don't get this on my Mac. How can I get rid of these messages?

0 votes

Per process memory monitoring tool for centos

I am looking for a tool which let me monitor memory consumption per process on CentOS 5/6. The tool should be able to save its history so I could see what amount of memory was consumed yesterday/week ago/etc by each process.

Can you recommend anything like that ?

+1 vote

Too many python installations. Should i remove them all and install the latest?

I'am using CentOS v6.4 on my VPS and hence 'yum' install manager and i just tried:

Code:
root@nikos [~]# which python
/usr/bin/python
root@nikos [~]# which python3
/root/.local/lib/python2.7/bin/python3
root@nikos [~]# which python3.3
/root/.local/lib/python2.7/bin/python3.3
root@nikos [~]#

Why so many pythons in my system.
Now in the case of my Python3 installation, it looks like i have two parallel installations of Python3, but i don't. One is almost certainly a symlink to the other and not an actual installation.

I'm thinking of:

yum remove python
yum remove python3
yum remove python3.3

and

yum install python3.3.2 from scratch.

I'm sceptic about uninstalling python 2.x though. Seems to me as a bad idea because most of the core system utilities are written in Python 2.6+. Yum, for example, is a collection of Python 2.6 programs. If i actually do "yum remove python" i will see most of my core system get listed in the uninstall dependency list -- which is a Bad Thing.

But then again i dont like the idea of having too many Python into my system.

0 votes

How do I debug why Centos is asking for fonts to be installed?

Quite a few times a day, both the latest Pan & Firefox available in standard CentOS repositories elicit what
(now) appears to be a CentOS request to install fonts,e.g.:

http://i39.tinypic.com/2i7qm2p.png

This latest message says: Pan newsreader wants to install fonts
- Thai
- Cherokee

At first I had thought it was Pan and Firefox causing these messages, but I was just told it's actually CentOS which is making these requests.

Hence, I ask:
Q: How do I debug why these requests are being made?
Q: How do I prevent these messages from popping up?

Why on CentOS, python consumes too much memory ?

Your comment on this post:

1 Answer

Your comment on this answer:

Your answer

Preview