Python中的最佳实践

[BruceChen7]

参考资料

• python3 import
• wtfpython
• Good logging practice in Python – Fang-Pen's coding note
• Python 之日志处理（logging 模块） - 云游道士 - 博客园
• fluent-python

项目组织

project/
    main.py
    sub1/
        __init__.py
        helper.py
    sub2/
        __init__.py
        utils.py
    tests/
        __init__.py
        test_sub1.py
        test_sub2.py

• 运行一个测试例子

  python -m tests.test_sub1

• 注意不是tests/test_sub1
• running a single test case or test method.
• Also you can run a single TestCase or a single test method:

  $ python -m tests.test_sub1.SampleTestCase
  $ python -m tests.test_sub1.SampleTestCase.test_method

• Running all tests
  • One way is to use unittest discovery mentioned in the A Typical directory structure for running tests using unittest, as copied below:

    $ cd new_project
    $ python -m unittest discover

  • This will run all the test*.py modules inside the tests package.

is 和 ==

>>> a = 256
>>> b = 256
>>> a is b
True

>>> a = 257
>>> b = 257
>>> a is b
False

>>> a = []
>>> b = []
>>> a is b
False

>>> a = tuple()
>>> b = tuple()
>>> a is b
True

• is 用来 check 两个对象是否是同一个，能理解为是同一个地址的对象，是引用的测试

• == 是用来比较两个对象的值

• 当启动 python 时候，会分配 -5 到 256 的对象。

• 这是因为这些值使用的很频繁

• The current implementation keeps an array of integer objects for all integers between -5 and 256, when you create an int in that range you just get back a reference to the existing object.

• So it should be possible to change the value of 1. I suspect the behavior of Python, in this case, is undefined

  >>> id(256)
  10922528
  >>> a = 256
  >>> b = 256
  >>> id(a)
  10922528
  >>> id(b)
  10922528
  >>> id(257)
  140084850247312
  >>> x = 257
  >>> y = 257
  >>> id(x)
  140084850247440
  >>> id(y)
  140084850247344

• 同样的优化适应于不可变的对象，比如空 tuple，而 list 是可变的，所以 is 判断为假。

• Both a and b refer to the same object when initialized with same value in the same line.

  >>> a, b = 257, 257
  >>> id(a)
  140640774013296
  >>> id(b)
  140640774013296
  >>> a = 257
  >>> b = 257
  >>> id(a)
  140640774013392
  >>> id(b)
  140640774013488

hash

some_dict = {}
some_dict[5.5] = "JavaScript"
some_dict[5.0] = "Ruby"
some_dict[5] = "Python"
>> some_dict[5.5]
"JavaScript"
>>> some_dict[5.0] # "Python" destroyed the existence of "Ruby"?
"Python"
>>> some_dict[5]
"Python"

>>> complex_five = 5 + 0j
>>> type(complex_five)
complex
>>> some_dict[complex_five]
"Python"

• python 中字典的相等性测试是 by equivalence, not identity. 尽管 5，5.0，5 + 0j 都是不同对象，但是都相等

  >>> 5 == 5.0 == 5 + 0j
  True
  >>> 5 is not 5.0 is not 5 + 0j
  True
  >>> some_dict = {}
  >>> some_dict[5.0] = "Ruby"
  >>> 5.0 in some_dict
  True
  >>> (5 in some_dict) and (5 + 0j in some_dict)
  True

内心深处，都一样

class WTF:
  pass
>>> WTF() == WTF() # two different instances can't be equal
False
>>> WTF() is WTF() # identities are also different
False
>>> hash(WTF()) == hash(WTF()) # hashes _should_ be different as well
True
>>> id(WTF()) == id(WTF())
True

• 未完待续

keep trying..

def some_func():
    try:
        return 'from_try'
    finally:
        return 'from_finally'

def another_func():
    for _ in range(3):
        try:
            continue
        finally:
            print("Finally!")

def one_more_func(): # A gotcha!
    try:
        for i in range(3):
            try:
                1 / i
            except ZeroDivisionError:
                # Let's throw it here and handle it outside for loop
                raise ZeroDivisionError("A trivial divide by zero error")
            finally:
                print("Iteration", i)
                break
    except ZeroDivisionError as e:
        print("Zero division error occurred", e)

>>> some_func()
'from_finally'

>>> another_func()
Finally!
Finally!
Finally!

>>> 1 / 0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero

>>> one_more_func()
Iteration 0

• 当 return，continue，break 语句在 try...finally 中使用的时候，finally 从句也会在退出的时候执行
• 一个函数最后退出的值，取决于最后一个 return 语句。
• 当 finally 语句执行 return 和 break 的时候，中间产生异常会被丢弃。

for

some_string = "wtf"
some_dict = {}
for i, some_dict[i] in enumerate(some_string):
    i = 10
>>> some_dict # An indexed dict appears.
{0: 'w', 1: 't', 2: 'f'}

• 和这个类比：

  for i in range(4):
      print(i)
      i = 10

• for 语句的定义：
  • for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]
  • {exprlist} = {next_value} 在每次迭代的时候都会呗执行。上面的代码执行：

    >>> i, some_dict[i] = (0, 'w')
    >>> i, some_dict[i] = (1, 't')
    >>> i, some_dict[i] = (2, 'f')
    >>> some_dict

list 坑

# Let's initialize a row
row = [""] * 3 #row i['', '', '']
# Let's make a board
board = [row] * 3

>>> board
[['', '', ''], ['', '', ''], ['', '', '']]
>>> board[0]
['', '', '']
>>> board[0][0]
''
>>> board[0][0] = "X"
>>> board
[['X', '', ''], ['X', '', ''], ['X', '', '']]

能这样避免

>>> board = [['']*3 for _ in range(3)]
>>> board[0][0] = "X"
>>> board
[['X', '', ''], ['', '', ''], ['', '', '']]

output function

funcs = []
results = []
for x in range(7):
    def some_func():
        return x
    funcs.append(some_func)
    results.append(some_func())  # note the function call here
funcs_results = [func() for func in funcs]

>>> results
[0, 1, 2, 3, 4, 5, 6]
>>> funcs_results
[6, 6, 6, 6, 6, 6, 6]

>>> powers_of_x = [lambda x: x**i for i in range(10)]
>>> [f(2) for f in powers_of_x]
[512, 512, 512, 512, 512, 512, 512, 512, 512, 512]

• When defining a function inside a loop that uses the loop variable in its body, the loop function's closure is bound to the variable, not its value.
• So all of the functions use the latest value assigned to the variable for computation.
• To get the desired behavior you can pass in the loop variable as a named variable to the function.
• Why does this work? Because this will define the variable again within the function's scope.

funcs = []
for x in range(7):
    def some_func(x=x):
        return x
    funcs.append(some_func)
>>> funcs_results = [func() for func in funcs]
>>> funcs_results
[0, 1, 2, 3, 4, 5, 6]

list 删除

list_1 = [1, 2, 3, 4]
list_2 = [1, 2, 3, 4]
list_3 = [1, 2, 3, 4]
list_4 = [1, 2, 3, 4]

for idx, item in enumerate(list_1):
    del item

for idx, item in enumerate(list_2):
    list_2.remove(item)

for idx, item in enumerate(list_3[:]):
    list_3.remove(item)

for idx, item in enumerate(list_4):
    list_4.pop(idx)

>>> list_1
[1, 2, 3, 4]
>>> list_2
[2, 4]
>>> list_3
[]
>>> list_4
[2, 4]

• 在遍历的过程中改变对象不是好主意。
• 好的方式是通过迭代拷贝的对象，正如 list_3[:]所做的。

  >>> some_list = [1, 2, 3, 4]
  >>> id(some_list)
  139798789457608
  >>> id(some_list[:]) # Notice that python creates new object for sliced list.
  139798779601192

del remove 和 pop 的区别

• del var_name 就是 removes the binding of the var_name from the local or global namespace (That's why the list_1 is unaffected).
• remove removes the first matching value, not a specific index, raises ValueError if the value is not found.
• pop removes the element at a specific index and returns it, raises IndexError if an invalid index is specified

Why the output is [2, 4]?

• 迭代开始的时候，删除了 1，然后 list 变成了 2，3，4，这样 2 在 index0，上，下一次删除 index 1 的时候变成删除 3.

Loop variables leaking out!

for x in range(7):
    if x == 6:
        print(x, ': for x inside loop')
print(x, ': x in global')

6 : for x inside loop
6 : x in global

# This time let's initialize x first
x = -1
for x in range(7):
    if x == 6:
        print(x, ': for x inside loop')
print(x, ': x in global')

# python 2
>>> x = 1
>>> print([x for x in range(5)])
[0, 1, 2, 3, 4]
>>> print(x)
4

# python3
>>> x = 1
>>> print([x for x in range(5)])
[0, 1, 2, 3, 4]
>>> print(x)
1

• In Python, for-loops use the scope they exist in and leave their defined loop-variable behind.
• This also applies if we explicitly defined the for-loop variable in the global namespace before.
• In this case, it will rebind the existing variable.

Beware of default mutable arguments!

def some_func(default_arg=[]):
    default_arg.append("some_string")
    return default_arg
>>> some_func()
['some_string']
>>> some_func()
['some_string', 'some_string']
>>> some_func([])
['some_string']
>>> some_func()
['some_string', 'some_string', 'some_string']


>> some_func.__defaults__ #This will show the default argument values for the function
([],)
>>> some_func()
>>> some_func.__defaults__
(['some_string'],)
>>> some_func()
>>> some_func.__defaults__
(['some_string', 'some_string'],)
>>> some_func([])
>>> some_func.__defaults__
(['some_string', 'some_string'],)

使用 None 来避免

def some_func(default_arg=None):
    if default_arg is None:
        default_arg = []
    default_arg.append("some_string")
    return default_arg

Catching the Exceptions

some_list = [1, 2, 3]
try:
    # This should raise an ``IndexError``
    print(some_list[4])
except IndexError, ValueError:
    print("Caught!")

try:
    # This should raise a ``ValueError``
    some_list.remove(4)
except IndexError, ValueError:
    print("Caught again!")

# output 2.x
Caught!

ValueError: list.remove(x): x not in list

# output 3.x
  File "<input>", line 3
    except IndexError, ValueError:
                     ^
SyntaxError: invalid syntax

对于 python2

some_list = [1, 2, 3]
try:
   # This should raise a ``ValueError``
   some_list.remove(4)
except (IndexError, ValueError), e:
   print("Caught again!")
   print(e)

对于 python3

some_list = [1, 2, 3]
try:
    some_list.remove(4)
except (IndexError, ValueError) as e:
    print("Caught again!")
    print(e)

Wild imports *

# File: module.py

def some_weird_name_func_():
    print("works!")

def _another_weird_name_func():
    print("works!")

>>> from module import *
>>> some_weird_name_func_()
"works!"
>>> _another_weird_name_func()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name '_another_weird_name_func' is not defined

• 通常不会使用*号来导入包，因为一是使用_开始的名字不能导入
• from module import a, b, c

>>> from module import some_weird_name_func_, _another_weird_name_func
>>> _another_weird_name_func()
works!

• 使用__all__来让 import * 可用

__all__ = ['_another_weird_name_func']

def some_weird_name_func_():
    print("works!")

def _another_weird_name_func():
    print("works!")


>>> _another_weird_name_func()
"works!"
>>> some_weird_name_func_()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'some_weird_name_func_' is not defined

common

# test for hex function
num = 16
print (hex(num)) # 0x10 convert a integer num to hex number;
num = '1'
print(int(num)) # convert a string or float num into int number
print(float(num))# convert a string or int num into float number

# test 2 for global num and id() function
num = 20
def fun():
    num = 100
    print id(num) #the num object in this function is different from the num outside the function,so the address is different
    print num
fun()
print num # 20
print id(num) #id function return the address of the object

# test 3 for global num
def global_num():
    global num # declare using a global num
    num = 10
    print num
    print id(num)

num = 100
global_num()
print id(num) # two address is the same
print num

# test 4 use of len function
name = ['bruce','Li','Chen']
name1 = ('bruce',)
name2 = {'bruce':80,'Chen':100}
print len(name)# return the length of the name list
print len(name1)
print len(name)

# test 5 use of locals function
def locals_test():
    num = 1;
    num2 = 2
    print locals()
    print len(locals())
locals_test() # return {'num':1,'num2':2} <2></2>

# test 6 use of list function
# the same with tuple function

tmp = ['hello','world',1]
copy_tmp = list(tmp)
print copy_tmp #['hello', 'world', 1]
copy_tmp = list()
print copy_tmp  #[]

# test 7 use of raw_input function
s = raw_input('--> ') #--> write to the standard output
print s; #if you input 'Hello world',then output 'hello world'

# use of open function and try...except
try:
    f = open('a.txt','r')  # return a file openjec
except IOError:
    print 'can\'t find the file'
else:
    print 'Other errors happen'

# test 8 use of dir function  return the list of names in the current local scope.
class Teacher:

    def __init__(self,name,age):
        self.name = name
        self.age = age

instance = Teacher('bruce',12)
print dir(instance)  #['__doc__', '__init__', '__module__', 'age', 'name']

# use of max function
fruit = ['apple','banala','juice']
num = [1,2,3]
print max(num)
print max(fruit)

# use of type function
fruit = "apple"
num = 1
print type(fruit) #<type 'str'>
print type(num) #<type 'int


def TryExcept():
    try:
        ...
    except Exception as e:
        print e

文件操作

os.environ["HOME"] # 主目录
os.path.exist(file_name) #文件是否存

字符串操作

';'.join(['1', '2', '3'])
# '1;2;3'

# cat test.txt
# 111
# 222
# 333

f=open('test.txt', 'r')
';'.join(f)
# '111\n;222\n;333\n' #文件内容的 join

# 关键字格式化
"hello %(name)s" % {'name': 'word', 'key2': 'val2'}
# 'hello word'

"hello {name}".format(**{'name': 'word'})


# 行分隔

'Line 1\n\nLine 3\rLine 4\r\n'.splitlines()
# ['Line 1', '', 'Line 3', 'Line 4']


# 字符分隔
'1,2,3'.split(',', 1)
# ['1', '2,3']

 '1,2,3'.rsplit(',', 1)
# ['1,2', '3']

## 重复多次
[1, 2, 3] * 2
# [1, 2, 3, 1, 2, 3]
for i, v in enumerate(['a', 'b', 'c']):
     print i, v
# 0 a
# 1 b
# 2 c

# 第二个参数，可指定开始迭代的索引起始值
for i, v in enumerate(['a', 'b', 'c'], 1):
     print i, v
# 1 a
# 2 b
# 3 c


## 查找
# 返回 templates 列表中，template_id 字段为 2 的元素 (字典)
(item for item in templates if item["template_id"] == 2).next()

# 上面语句在找不到对应元素时会抛出 StopIteration 异常，如果不希望抛异常
# 能使用 next() 内置函数指定默认值
next((item for item in templates if item["template_id"] == 2), None)
# 找不到时，返回 None

set

## 生成式
{i for i in [1, 2, 2, 3]}


#实际上{1,2,3} 相当于 set([1,2,3])

a = {1, 2, 3}
a.remove(4)    # 4 不在 set 抛 KeyError 异常
a.discard(4)     # 不会抛出异常


# 添加集合项
s = {3, 4, 5}
s.update([1,2,3])
print s
# set([1, 2, 3, 4, 5])

s <= r # 相当于 s.issubset(r) s 是否为 r 的子集
s >= r # 相当于 s.issuperset(r) s 是否为 r 的超集
s < r #s 是否为 r 的真子集
s > r #s 是否为 r 的真超集
s | r # 相当于 s.union(r) 返回 s 和 r 的并集
s & t # 相当于 s.intersection(t) 返回 s 和 r 的交集
s - t # 相当于 s.difference(t) 返回 s 和 t 的差集（s 有但是 t 没有的元素）
s ^ t # 相当于 s.symmetric_difference(t) 返回个新集合，是 s 或 t 的成员，但不是 s 和 t 共有的成员

字典

d = {'a': 1}
d.update({'b': 2, 'c': 3})
# 更好看的写法
d.update(b=2, c=3)

## 默认设置字典的值，没有则改变，否则不改变
d = {'a': 1, 'b': 2}
d.setdefault('a', 2)
d.setdefault('c', 3)
print d
{'a': 1, 'c': 3, 'b': 2}

# defaultdict
strings = ('a', 'b', 'c', 'a','d', 'c', 'c', 'a')
# 如果 counts 的 key 不存在，则给定一个默认值 0
counts = defaultdict(int)

# 词频统计
for kw in strings:
    counts[kw] += 1

## python 的字典是无序的，OrderedDict 能保留 key 的顺序信息
## OrderedDict 的 Key 会按照插入的顺序排列

od = OrderedDict()
od['a'] = 1
od['b'] = 2
od.keys()
# 按照插入的 Key 的顺序返回
# ['z', 'y', 'x']

如何创建一个字典和给字典赋值

创建一个字典，和给字典赋值都是比较简单的，按照下面，即可创建。

>>> dict1 = {}
>>> dict2 = {'name': 'earth','port': 80}

当然，也能使用工厂方法来创建

>>> fdict = dict((['x',1],['y',2]))
>>>  fdict
{'y':2,'x':1}
>>> fdict = dict(name= "hello",sex ="male")
{"name":"hello","sex":"male"}

访问字典中的值

要遍历一个字典，使用in操作，然后通过字典键加上括号来访问其值

>>> dict2 = {'name': 'earth','port': 80}
>>> for key in dict2
>>>       print 'key= %s,value = %s' %(key,dict2[key])

如果访问一个不存在的键将会报错。

如何更新字典

能有几种方式对一个字典做修改：添加一个数据项或者是新元素，修改一个已经存在的数据项，删除一个已经存在的数据项。

>>>dict2['name'] = 'venus'
>>>dict2['port'] = 6969

如果字典中该键已经存在，则字典中该键对应的值将被新值替代。

删除字典某元素和字典

通常删除整个字典的操作是不常见的，下面记录了删除的代码：

del dict2['name']#删除name的条目
dict2.clear()  #删除整个条目

字典方法文档搜索

• 在 python 字典中输入：dict.xxx。

字典方法

构造字典对象

字典构造

字典方法

• len(dict) 获取字典项目
• clear() 清除所有的项
• copy() 浅拷贝复制
• fromkeys(seq,[,value]) 创建一个新的字典，键来自 seq，值设置为 value。
• get(key[,default]) 获取某个键的值，如果没有该键，默认返回 None，当然你能改变默认值
• items() 将所有的字典项以列表的方式返回，其中每一项都是来自于键和值。（这个键和值共同组成元祖）但在返回的时候，没有特殊顺序。
• iteritems() 的作用相同，但是返回的是迭代器对象而不是列表。
• keys() 将字典中的键以列表的方式返回。
• pop(key[,default]) 如果键值存在，那么直接删除该键 - 值对，并且返回其值，如果 default 没有给，并且键不存在，那么会产生 KeyError

列表

## 生成式

a = [1, 2, 3, 4, 5, 6]
['%02d' %i for i in a]
# ['01', '02', '03', '04', '05', '06']

# 区别 if 的位置
[ expression for item in list if conditional ]
[ val1 if conditional else val2 for item in list]
# 上面其实是 python 的类三目运算符例子

[ i for i in [1, 2, 3] if i >= 3]   [3]


## 多层嵌套
l = [[1,3],[5,7,6]]
[item for sublist in l for item in sublist]
#  [1, 3, 5, 7, 6]
[item for sublist in l for item in sublist if item >3]
# [5, 7, 6]

# 列表倒序
a[::-1]

itertools 操作

import itertools
x = itertools.accumulate(range(10))  # 对 range 迭代器进行累加
print(list(x)) # [0, 1, 3, 6, 10, 15, 21, 28, 36, 45]

# 连接多个迭代器
x = itertools.chain(range(3), range(4), [3,2,1])
print(list(x)) # [0, 1, 2, 0, 1, 2, 3, 3, 2, 1]

# 求列表或生成器中指定数目的元素不重复的所有组合
x = itertools.combinations(range(4), 3)
print(list(x)) # [(0, 1, 2), (0, 1, 3), (0, 2, 3), (1, 2, 3)]

# 按照真值表筛选元素
x = itertools.compress(range(5), (True, False, True, True, False))
print(list(x)) # [0, 2, 3]

# 保留对应真值为 False 的元素
x = itertools.filterfalse(lambda e: e < 5, (1, 5, 3, 6, 9, 4))
print(list(x)) # [5, 6, 9]

# 按照分组函数的值对元素进行分组
x = itertools.groupby(range(10), lambda x: x < 5 or x > 8)
for condition, numbers in x:
    print(condition, list(numbers)) # True [0, 1, 2, 3, 4]  False [5, 6, 7, 8]   True [9]

# 产生指定数目的元素的所有排列 (顺序有关)
x = itertools.permutations(range(4), 3)
print(list(x))

#
x = itertools.repeat(0, 5)
print(list(x)) #[0, 0, 0, 0, 0]

# 按照真值函数丢弃掉列表和迭代器前面的元素
x = itertools.dropwhile(lambda e: e < 5, range(10))
print(list(x)) # [5, 6, 7, 8, 9]

# 与 dropwhile 相反，保留元素直至真值函数值为假。
x = itertools.takewhile(lambda e: e < 5, range(10))
print(list(x)) # [0, 1, 2, 3, 4]

#
x = itertools.zip_longest(range(3), range(5))
y = zip(range(3), range(5))
print(list(x)) #[(0, 0), (1, 1), (2, 2), (None, 3), (None, 4)]
print(list(y)) #[(0, 0), (1, 1), (2, 2)]

collections

## dequed 的的使用使用
from collections import deque

# ctor which use iterable data
d = deque((1, 2, 3))
d1  = deque([1, 2, 3])
d2 = deque(range(10))

# append data to the end of deque
d.append(4) # 1, 2, 3, 4
d.appendleft(0) # 0, 1, 2, 3, 4

# count the number of deque number which equals 4
d.count(4) # 1

d.pop() # remove and return the right side of the deque
p.popleft() # remove and return the element of left side
# extend the right side of the deque
d.extend(iterable)

match 表达式

match variable:
    case value1:
        # execute code if the value of the variable matches value1
    case value2:
        # execute code if the value of the variable matches value2
    ...
    case _:
        # execute code if none of the above cases match

def interp_exp(e):
    # 是 python3.10 的高级特性
    match e:
        case BinOp(left, Add(), right):
            # 获取左右子表达式的值
            l = interp_exp(left)
            r = interp_exp(right)
            return add64(l, r)
        # 减法
        case BinOp(left, Sub(), right):
            l = interp_exp(left)
            r = interp_exp(right)
            return sub64(l, r)
        case UnaryOp(USub(), v):
            return neg64(interp_exp(v))
        # 常量值
        case Constant(value):
            return value
        # 遇到模式
        case Call(Name("input_int"), []):
            # 返回输出
            return input_int()
        case _:
            raise Exception("error in interp_exp, unexpected " + repr(e))

打印 requests_oauthlib 库调试日志

log = logging.getLogger('requests_oauthlib')
log.addHandler(logging.StreamHandler(sys.stdout))
log.setLevel(logging.DEBUG)

#type/python #public