??xml version="1.0" encoding="utf-8" standalone="yes"?>亚洲欧洲美洲综合色网,国产成人综合精品三级,亚洲大全视频http://www.aygfsteel.com/yglwxl/category/39566.htmlzh-cnMon, 30 Nov 2009 11:11:11 GMTMon, 30 Nov 2009 11:11:11 GMT60How does Python manage memory?(http://docs.python.org/3.1/faq/design.html)http://www.aygfsteel.com/yglwxl/archive/2009/11/30/304232.html?ji)?/dc:creator>?ji)?/author>Mon, 30 Nov 2009 03:01:00 GMThttp://www.aygfsteel.com/yglwxl/archive/2009/11/30/304232.htmlhttp://www.aygfsteel.com/yglwxl/comments/304232.htmlhttp://www.aygfsteel.com/yglwxl/archive/2009/11/30/304232.html#Feedback0http://www.aygfsteel.com/yglwxl/comments/commentRss/304232.htmlhttp://www.aygfsteel.com/yglwxl/services/trackbacks/304232.htmlThe details of Python memory management depend on the implementation. The standard C implementation of Python uses reference counting to detect inaccessible objects, and another mechanism to collect reference cycles, periodically executing a cycle detection algorithm which looks for inaccessible cycles and deletes the objects involved. The gc module provides functions to perform a garbage collection, obtain debugging statistics, and tune the collector’s parameters.

Jython relies on the Java runtime so the JVM’s garbage collector is used. This difference can cause some subtle porting problems if your Python code depends on the behavior of the reference counting implementation.

Sometimes objects get stuck in tracebacks temporarily and hence are not deallocated when you might expect. Clear the tracebacks with:

import sys
sys.exc_clear()
sys.exc_traceback = sys.last_traceback = None

Tracebacks are used for reporting errors, implementing debuggers and related things. They contain a portion of the program state extracted during the handling of an exception (usually the most recent exception).

In the absence of circularities and tracebacks, Python programs need not explicitly manage memory.

Why doesn’t Python use a more traditional garbage collection scheme? For one thing, this is not a C standard feature and hence it’s not portable. (Yes, we know about the Boehm GC library. It has bits of assembler code for most common platforms, not for all of them, and although it is mostly transparent, it isn’t completely transparent; patches are required to get Python to work with it.)

Traditional GC also becomes a problem when Python is embedded into other applications. While in a standalone Python it’s fine to replace the standard malloc() and free() with versions provided by the GC library, an application embedding Python may want to have its own substitute for malloc() and free(), and may not want Python’s. Right now, Python works with anything that implements malloc() and free() properly.

In Jython, the following code (which is fine in CPython) will probably run out of file descriptors long before it runs out of memory:

for file in <very long list of files>:
f = open(file)
c = f.read(1)

Using the current reference counting and destructor scheme, each new assignment to f closes the previous file. Using GC, this is not guaranteed. If you want to write code that will work with any Python implementation, you should explicitly close the file; this will work regardless of GC:

for file in <very long list of files>:
f = open(file)
c = f.read(1)
f.close()


]]>
Python 循环引用D内存泄漏的内存管理TEST ref fromQhttp://meizhini.javaeye.com/blog/249716http://www.aygfsteel.com/yglwxl/archive/2009/11/30/304229.html?ji)?/dc:creator>?ji)?/author>Mon, 30 Nov 2009 02:47:00 GMThttp://www.aygfsteel.com/yglwxl/archive/2009/11/30/304229.htmlhttp://www.aygfsteel.com/yglwxl/comments/304229.htmlhttp://www.aygfsteel.com/yglwxl/archive/2009/11/30/304229.html#Feedback0http://www.aygfsteel.com/yglwxl/comments/commentRss/304229.htmlhttp://www.aygfsteel.com/yglwxl/services/trackbacks/304229.html     因ؓ Python 有了自动垃圾回收功能Q不初学者就认ؓ自己从此q上了好日子Q不必再受内存泄漏的骚扰了。但如果查看一?Python 文?__del__() 函数的描qͼq道好日子里也是有阴云的。下面摘抄一Ҏ(gu)档内容:
    Some common situations that may prevent the reference count of an object from going to zero include: circular references between objects (e.g., a doubly-linked list or a tree data structure with parent and child pointers); a reference to the object on the stack frame of a function that caught an exception (the traceback stored in sys.exc_traceback keeps the stack frame alive); or a reference to the object on the stack frame that raised an unhandled exception in interactive mode (the traceback stored in sys.last_traceback keeps the stack frame alive).
    可见Q有 __del__() 函数的对象间的@环引用是D内存泄漏的主凶。特别说明:Ҏ(gu)?__del__() 函数?Python 对象间的循环引用Q是可以被自动垃圑֛收掉的?br />

?__del__() 函数的对象间的@环引用是D内存泄漏的主凶。特别说明:Ҏ(gu)?__del__() 函数?Python 对象间的循环引用Q是可以被自动垃圑֛收掉的。所以,没什么事Q千万不要轻易启用__del__操作?br /> 如何知道一个对象是否内存泄漏了呢?
    Ҏ(gu)一、当你认Z个对象应该被销毁时Q即引用计数?0Q,可以通过 sys.getrefcount(obj) 来获取对象的引用计数QƈҎ(gu)q回值是否ؓ 0 来判断是否内存泄漏。如果返回的引用计数不ؓ 0Q说明在此刻对象 obj 是不能被垃圾回收器回收掉的?br />     Ҏ(gu)二、也可以通过 Python 扩展模块 gc 来查看不能回收的对象的详l信息?br />

    首先Q来看一D|常的试代码Q?br />
#--------------- code begin --------------

# -*- coding: utf-8 -*-

import gc
import sys

class CGcLeak(object):
    def __init__(self):
        self._text = '#'*10

    def __del__(self):
        pass

def make_circle_ref():
    _gcleak = CGcLeak()
#   _gcleak._self = _gcleak # test_code_1
    print '_gcleak ref count0:%d' % sys.getrefcount(_gcleak)
    del _gcleak
    try:
        print '_gcleak ref count1:%d' % sys.getrefcount(_gcleak)
    except UnboundLocalError:
        print '_gcleak is invalid!'

def test_gcleak():
    # Enable automatic garbage collection.
    gc.enable()
    # Set the garbage collection debugging flags.
    gc.set_debug(gc.DEBUG_COLLECTABLE | gc.DEBUG_UNCOLLECTABLE | \
        gc.DEBUG_INSTANCES | gc.DEBUG_OBJECTS)

    print 'begin leak test...'
    make_circle_ref()

    print 'begin collect...'
    _unreachable = gc.collect()
    print 'unreachable object num:%d' % _unreachable
    print 'garbage object num:%d' % len(gc.garbage)

if __name__ == '__main__':
    test_gcleak()

#--------------- code end ----------------

    ?test_gcleak() 中,讄垃圾回收器调试标志后Q再?collect() q行垃圾回收Q最后打印出该次垃圾回收发现的不可达的垃圑֯象数和整个解释器中的垃圾对象数?br />
    gc.garbage 是一?list 对象Q列表项是垃圾收集器发现的不可达Q即是垃圑֯象)、但又不能释放(即不能回Ӟ的对象。文档描qCؓQA list of objects which the collector found to be unreachable but could not be freed (uncollectable objects).
    通常Qgc.garbage 中的对象是引用环中的对象。因?Python 不知道按照什么样的安全次序来调用环中对象?__del__() 函数Q导致对象始l存zd gc.garbage 中,造成内存泄漏。如果知道一个安全的ơ序Q那么就打破引用环,再执?del gc.garbage[:] Q以清空垃圾对象列表?br />
    上段代码输出为(#后字W串为笔者所加注释)Q?br /> #-----------------------------------------
begin leak test...
# 变量 _gcleak 的引用计Cؓ 2.
_gcleak ref count0:2
# _gcleak 变ؓ不可?unreachable)的非法变?
_gcleak is invalid!
# 开始垃圑֛?br /> begin collect...
# 本次垃圾回收发现的不可达的垃圑֯象数?0.
unreachable object num:0
# 整个解释器中的垃圑֯象数?0.
garbage object num:0
#-----------------------------------------
    可见 _gcleak 对象的引用计数是正确的,也没有Q何对象发生内存泄漏?br />
    如果不注释掉 make_circle_ref() 中的 test_code_1 语句Q?br />     _gcleak._self = _gcleak
也就是让 _gcleak 形成一个自己对自己的@环引用。再q行上述代码Q输出结果就变成Q?br /> #-----------------------------------------
begin leak test...
_gcleak ref count0:3
_gcleak is invalid!
begin collect...
# 发现可以回收的垃圑֯? 地址?012AA090Q类型ؓ CGcLeak.
gc: uncollectable <CGcLeak 012AA090>
gc: uncollectable <dict 012AC1E0>
unreachable object num:2
#!! 不能回收的垃圑֯象数?1Q导致内存泄漏!
garbage object num:1
#-----------------------------------------
    可见 <CGcLeak 012AA090> 对象发生了内存泄漏!Q而多出的 dict 垃圾是泄漏?_gcleak 对象的字典,打印出字怿息ؓ:
{'_self': <__main__.CGcLeak object at 0x012AA090>, '_text': '##########'}

    除了对自q循环引用Q多个对象间的@环引用也会导致内存泄漏。简单D例如下:

#--------------- code begin --------------

class CGcLeakA(object):
    def __init__(self):
        self._text = '#'*10

    def __del__(self):
        pass

class CGcLeakB(object):
    def __init__(self):
        self._text = '*'*10

    def __del__(self):
        pass

def make_circle_ref():
    _a = CGcLeakA()
    _b = CGcLeakB()
    _a._b = _b  # test_code_2
    _b._a = _a  # test_code_3
    print 'ref count0:a=%d b=%d' % \
        (sys.getrefcount(_a), sys.getrefcount(_b))
#   _b._a = None    # test_code_4
    del _a
    del _b
    try:
        print 'ref count1:a=%d' % sys.getrefcount(_a)
    except UnboundLocalError:
        print '_a is invalid!'
    try:
        print 'ref count2:b=%d' % sys.getrefcount(_b)
    except UnboundLocalError:
        print '_b is invalid!'

#--------------- code end ----------------

    q次试后输出结果ؓQ?br /> #-----------------------------------------
begin leak test...
ref count0:a=3 b=3
_a is invalid!
_b is invalid!
begin collect...
gc: uncollectable <CGcLeakA 012AA110>
gc: uncollectable <CGcLeakB 012AA0B0>
gc: uncollectable <dict 012AC1E0>
gc: uncollectable <dict 012AC0C0>
unreachable object num:4
garbage object num:2
#-----------------------------------------
可见 _a,_b 对象都发生了内存泄漏。因Z者是循环引用Q垃圑֛收器不知道该如何回收Q也是不知道该首先调用那个对象?__del__() 函数?br />
    采用以下MҎ(gu)Q打破环状引用,可以避免内存泄漏:
[1] 注释?make_circle_ref() 中的 test_code_2 语句Q?br /> [2] 注释?make_circle_ref() 中的 test_code_3 语句Q?br /> [3] 取消?make_circle_ref() 中的 test_code_4 语句的注释?br /> 相应输出l果变ؓQ?br /> #-----------------------------------------
begin leak test...
ref count0:a=2 b=3  # 注:此处输出l果视情况变?
_a is invalid!
_b is invalid!
begin collect...
unreachable object num:0
garbage object num:0
#-----------------------------------------


    l论QPython ?gc 有比较强的功能,比如讄 gc.set_debug(gc.DEBUG_LEAK) 可以进行@环引用导致的内存泄露的检查。如果在开发时q行内存泄露查;在发布时能够保不会内存泄露Q那么就可以廉 Python 的垃圑֛收时间间隔、甚至主动关闭垃圑֛收机Ӟ从而提高运行效率?br />

]]>
Come from : Python源码剖析——Python的内存管理机?/title><link>http://www.aygfsteel.com/yglwxl/archive/2009/11/30/304225.html</link><dc:creator>?ji)?/dc:creator><author>?ji)?/author><pubDate>Mon, 30 Nov 2009 02:35:00 GMT</pubDate><guid>http://www.aygfsteel.com/yglwxl/archive/2009/11/30/304225.html</guid><wfw:comment>http://www.aygfsteel.com/yglwxl/comments/304225.html</wfw:comment><comments>http://www.aygfsteel.com/yglwxl/archive/2009/11/30/304225.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.aygfsteel.com/yglwxl/comments/commentRss/304225.html</wfw:commentRss><trackback:ping>http://www.aygfsteel.com/yglwxl/services/trackbacks/304225.html</trackback:ping><description><![CDATA[1.块I间的内存池<br /> 在Python中,许多时候申L内存都是块的内存,q些块内存在申请后Q很快又?br /> 被释放,׃q些内存的申请ƈ不是Z创徏对象Q所以ƈ没有对象一U的内存池机制。这?br /> 意味着Python在运行期间会大量地执行malloc和free的操作,频繁地在用户态和核心态之?br /> q行切换Q这严重媄响Python的执行效率。ؓ了加速Python的执行效率,Python引入了一<br /> 个内存池机制Q用于管理对块内存的申请和释放。这也就是之前提到的Pymalloc机制.<br /> <br /> 2.在Python2.5中,Python内部默认的小块内存与大块内存的分界点定在256个字节,q个<br /> 分界点由前面我们看到的名为SMALL_REQUEST_THRESHOLD的符h制。也是_当申<br /> L内存于256字节ӞPyObject_Malloc会在内存池中甌内存Q当甌的内存大?56<br /> 字节ӞPyObject_Malloc的行为将蜕化为malloc的行为。当Ӟ通过修改Python源代码,?br /> 们可以改变这个默认|从而改变Python的默认内存管理行为?<br /> <br /> 3.在一个对象的引用计数减ؓ0Ӟ与该对象对应的析构函数就会被调用Q但是要特别注意的是Q调用析构函数ƈ不意味着最l一定会调用free释放内存I间Q如果真是这L话,那频J地甌、释攑ֆ存空间会?<span id="wmqeeuq" class="hilite1">Python</span>的执行效率大打折扣(更何?span class="hilite1">Python</span>已经多年背负了h们对其执行效率的不满Q。一般来_<span id="wmqeeuq" class="hilite1">Python</span>中大量采用了内存对象池的技术,使用q种技术可以避免频J地甌和释攑ֆ存空间。因此在析构Ӟ通常都是对象占用的I间归还到内存池中?br /> "q个问题是:<span id="wmqeeuq" class="hilite1">Python</span>的arena从来不释放pool。这个问题ؓ什么会引vcM于内存泄漏的现象呢。考虑q样一U情形,甌10*1024*1024?6字节的小内存Q这意味着必须使用160M的内存,׃<span id="wmqeeuq" class="hilite1">Python</span>没有默认前面提到的限制内存池的WITH_MEMORY_LIMITS~译W号打开Q所?span class="hilite1">Python</span>会完全用arena来满你的需求,q都没有问题Q关键的问题在于q了一D|_你将所有这?6字节的内存都释放了,q些内存都回到arena的控制中Q似乎没有问题。但是问题恰恰就在这时出C。因为arena始终不会释放它维护的pool集合Q所以这160M的内存始l被<span id="wmqeeuq" class="hilite1">Python</span>占用Q如果以后程序运行中再也不需?60M如此巨大的内存, <br /> q点内存岂不是就费了?" <br /> <span id="wmqeeuq" class="hilite1">python</span>内存理规则:<br /> del的时?把list的元素释放掉,把管理元素的大对象回收到py对象~冲池里. <br /> <img src ="http://www.aygfsteel.com/yglwxl/aggbug/304225.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.aygfsteel.com/yglwxl/" target="_blank">?ji)?/a> 2009-11-30 10:35 <a href="http://www.aygfsteel.com/yglwxl/archive/2009/11/30/304225.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>Python 块I间内存理TEST ref fromQ?http://www.javaeye.com/topic/309753http://www.aygfsteel.com/yglwxl/archive/2009/11/27/303916.html?ji)?/dc:creator>?ji)?/author>Fri, 27 Nov 2009 08:54:00 GMThttp://www.aygfsteel.com/yglwxl/archive/2009/11/27/303916.htmlhttp://www.aygfsteel.com/yglwxl/comments/303916.htmlhttp://www.aygfsteel.com/yglwxl/archive/2009/11/27/303916.html#Feedback1http://www.aygfsteel.com/yglwxl/comments/commentRss/303916.htmlhttp://www.aygfsteel.com/yglwxl/services/trackbacks/303916.htmldef test():
   for i in range ( 1000000 * 10 ):
        del i

if ( __name__ == "__main__" ):
    test()
    while ( True ):
        time.sleep( 1 )
观察mem:内存l持不变! 

从这点可以猜?python不是立即释放资源?

个h试代码:
-----------------------------------------------test0.py-------------------------------------

import time

def test():
    for i in range ( 1000000 * 10 ):
        del i


def test_2():
    #i = range ( 1000000 * 10 )
    #del i
    pass

def test_3():
    #i = "*" * ( 1000000 * 10 )
    #del i
    pass


if ( __name__ == "__main__" ):
    for i in range( 10 ):
        test()
        test_2()
        test_3()
        time.sleep( 1 )
    while ( True ):
        time.sleep( 1 )
-----------------------------------------------------test0.py-------------------------------------- 

q行 python test0.py

"while ( True ):
        time.sleep( 1 )
 "
保证python不退?

发现python的内存占用率?0%.

 

如何解决q个问题?看下面的:

-----------------------------------------------test1.py-------------------------------------

#coding=utf-8
import time

max_number = 1000000 * 10
def test_0():
    for i in range ( max_number ):
        del i

def test_1():
    for i in range( 1000 ):
        for i in range ( max_number / 1000 ):
            del i

if ( __name__ == "__main__" ):
    #test_0()#内存使用率占40%
    test_1()#内存使用率占0.2%
 
    print "---------------------"
    while ( True ):
        time.sleep( 1 )

-----------------------------------------------test1.py-------------------------------------

 我想问题:问题也许解决?

q就要看你的实际需求是什么了.

例如:

我做q一个爬虫程?如果不断往q个U程里面传递url,那么q个U程不到一会就挂了.我的改进Ҏ(gu):是控制q线E能够接受的url队列长度.以及其他的优?
 其实q个不是循环D的内存被python持有,而是range( n )让python分配了很多的内存.退出test(),python回收内存,但是pythonq不释放?而是让pool持有内存I间.



]]>
Python- 映射 listhttp://www.aygfsteel.com/yglwxl/archive/2009/05/14/270575.html?ji)?/dc:creator>?ji)?/author>Thu, 14 May 2009 03:57:00 GMThttp://www.aygfsteel.com/yglwxl/archive/2009/05/14/270575.htmlhttp://www.aygfsteel.com/yglwxl/comments/270575.htmlhttp://www.aygfsteel.com/yglwxl/archive/2009/05/14/270575.html#Feedback0http://www.aygfsteel.com/yglwxl/comments/commentRss/270575.htmlhttp://www.aygfsteel.com/yglwxl/services/trackbacks/270575.html >>>params.keys()
['a', 's', 'b']
>>>params.values()
['av', 'sv', 'bv']
>>>params.items()
[('a', 'av'), ('s', 'sv'), ('b', 'bv')]
>>>[k for k,v in params.items()]
['a', 's', 'b']
>>>[v for k,v in params.items()]
['av', 'sv', 'bv']
>>>["%s=%s" % (k,v) for k,v in params.items()]
['a=av', 's=sv', 'b=bv']

]]>
Python study- List extend() and append()http://www.aygfsteel.com/yglwxl/archive/2009/05/14/270562.html?ji)?/dc:creator>?ji)?/author>Thu, 14 May 2009 03:15:00 GMThttp://www.aygfsteel.com/yglwxl/archive/2009/05/14/270562.htmlhttp://www.aygfsteel.com/yglwxl/comments/270562.htmlhttp://www.aygfsteel.com/yglwxl/archive/2009/05/14/270562.html#Feedback0http://www.aygfsteel.com/yglwxl/comments/commentRss/270562.htmlhttp://www.aygfsteel.com/yglwxl/services/trackbacks/270562.html >>>li=['a','b']
>>>li.extend(['c','d'])
>>>li
['a', 'b', 'c', 'd']
>>>li.append(['e','f'])
>>>li
['a', 'b', 'c', 'd',['e','f']]

2. 搜烦 搜烦 list
>>>li.index('b')
>>>li
1

3. List q算W+ ?×
>>> li = ['a', 'b']
>>> li = li + ['example', 'new'] 
>>> li
['a', 'b', 'example', 'new']
>>> li += ['two']              
>>> li
['a', 'b', 'example', 'new', 'two']
>>> li = [1, 2] * 3      
>>> li
[1, 2, 1, 2, 1, 2]


4.何谓 Python 中的 True
·0为false;   其它所有数值皆?true?br /> ·IZ ("") 为false;  其它所有字W串皆ؓ true?br /> ·Ilist ([])为false;  其它所有字W串皆ؓ true?br /> ·I?tuple (()) 为false;  其它所有字W串皆ؓ true?br /> ·I?dictionary ({}) 为false;  其它所有字W串皆ؓ true?

]]>
վ֩ģ壺 | | ˮ| ϻ| Խ| | | | ߰| Ѱ| Զ| °Ͷ| Ͽ| | | | | | ɽ| | | | ˮ| | | ˮ| Ȫ| | | ͨ| Ϫ| | ¹| | ϲ| | | | ˹| ӱ| |