This package contains a pure-Python MySQL client library. The goal of PyMySQL is to be a drop-in replacement for MySQLdb and work on CPython, PyPy and IronPython.
如源码：
self.rowcount = sum(self.execute(query, arg) for arg in args)executemany是对execute的包装，并非multiple rows，因此在做insert操作的时候，数据量稍大些，性能就变得很差。之前一直没发现这个问题，以为executemany就是批量入库，从而浪费了不少时间与资源。 而如果利用execute并发挥MySQL的multiple rows作用，同样12万条数据入库，能从近6000秒提升至20秒。
优化前：
sql = "INSERT INTO mtable(field1, field2, field3...) VALUES (%s, %s, %s...)"
for item in datas:
batch_list.append([v1, v2, v3...])
# 批量插入
if len(batch_list) == common.mysql_batch_num:
cur.executemany(sql, batch_list)
conn.commit()
batch_list = []
counts += len(batch_list)
common.print_log("inserted:" + str(counts))优化后：
sql = "INSERT INTO mtable(field1, field2, field3...) VALUES (%s, %s, %s...)"
for item in datas:
batch_list.append(common.multipleRows([v1, v2, v3...]))
# 批量插入
if len(batch_list) == common.mysql_batch_num:
sql = "INSERT INTO mtable(field1, field2, field3...) VALUES %s " % ','.join(batch_list)
cur.execute(sql)
conn.commit()
batch_list = []
counts += len(batch_list)
common.print_log("inserted:" + str(counts))
# 返回可用于multiple rows的sql拼装值
def multipleRows(params):
ret = []
# 根据不同值类型分别进行sql语法拼装
for param in params:
if isinstance(param, (int, long, float, bool)):
ret.append(str(param))
elif isinstance(param, (str, unicode)):
ret.append('"' + param + '"')
else:
print_log('unsupport value: %s ' % param)
return '(' + ','.join(ret) + ')'- Python -- one of the following:
- CPython >= 2.6 or >= 3.3
- PyPy >= 4.0
- IronPython 2.7
- MySQL Server -- one of the following:
The last stable release is available on PyPI and can be installed with pip:
$ pip install PyMySQL
Alternatively (e.g. if pip is not available), a tarball can be downloaded
from GitHub and installed with Setuptools:
$ # X.X is the desired PyMySQL version (e.g. 0.5 or 0.6). $ curl -L https://github.com/PyMySQL/PyMySQL/tarball/pymysql-X.X | tar xz $ cd PyMySQL* $ python setup.py install $ # The folder PyMySQL* can be safely removed now.
If you would like to run the test suite, create database for test like this:
mysql -e 'create database test_pymysql DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;' mysql -e 'create database test_pymysql2 DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;'
Then, copy the file .travis.databases.json to pymysql/tests/databases.json
and edit the new file to match your MySQL configuration:
$ cp .travis.databases.json pymysql/tests/databases.json $ $EDITOR pymysql/tests/databases.json
To run all the tests, execute the script runtests.py:
$ python runtests.py
A tox.ini file is also provided for conveniently running tests on multiple
Python versions:
$ tox
The following examples make use of a simple table
CREATE TABLE `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`email` varchar(255) COLLATE utf8_bin NOT NULL,
`password` varchar(255) COLLATE utf8_bin NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin
AUTO_INCREMENT=1 ;import pymysql.cursors
# Connect to the database
connection = pymysql.connect(host='localhost',
user='user',
password='passwd',
db='db',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
try:
with connection.cursor() as cursor:
# Create a new record
sql = "INSERT INTO `users` (`email`, `password`) VALUES (%s, %s)"
cursor.execute(sql, ('webmaster@python.org', 'very-secret'))
# connection is not autocommit by default. So you must commit to save
# your changes.
connection.commit()
with connection.cursor() as cursor:
# Read a single record
sql = "SELECT `id`, `password` FROM `users` WHERE `email`=%s"
cursor.execute(sql, ('webmaster@python.org',))
result = cursor.fetchone()
print(result)
finally:
connection.close()This example will print:
{'password': 'very-secret', 'id': 1}DB-API 2.0: http://www.python.org/dev/peps/pep-0249
MySQL Reference Manuals: http://dev.mysql.com/doc/
MySQL client/server protocol: http://dev.mysql.com/doc/internals/en/client-server-protocol.html
PyMySQL mailing list: https://groups.google.com/forum/#!forum/pymysql-users
PyMySQL is released under the MIT License. See LICENSE for more information.