Public speaking course notes Read "Dynamo, Amazon’s Highly Available Key-value Store" Read "Bigtable, A Distributed Storage System for Structured Data" Read "Streaming Systems" 3, Watermarks Read "Streaming Systems" 1&2, Streaming 101 Read "F1, a distributed SQL database that scales" Read "Zanzibar, Google’s Consistent, Global Authorization System" Read "Spanner, Google's Globally-Distributed Database" Read "Designing Data-intensive applications" 12, The Future of Data Systems IOS development with Swift Read "Designing Data-intensive applications" 10&11, Batch and Stream Processing Read "Designing Data-intensive applications" 9, Consistency and Consensus Read "Designing Data-intensive applications" 8, Distributed System Troubles Read "Designing Data-intensive applications" 7, Transactions Read "Designing Data-intensive applications" 6, Partitioning Read "Designing Data-intensive applications" 5, Replication Read "Designing Data-intensive applications" 3&4, Storage, Retrieval, Encoding Read "Designing Data-intensive applications" 1&2, Foundation of Data Systems Three cases of binary search TAMU Operating System 2 Memory Management TAMU Operating System 1 Introduction Overview in cloud computing 2 TAMU Operating System 7 Virtualization TAMU Operating System 6 File System TAMU Operating System 5 I/O and Disk Management TAMU Operating System 4 Synchronization TAMU Operating System 3 Concurrency and Threading TAMU Computer Networks 5 Data Link Layer TAMU Computer Networks 4 Network Layer TAMU Computer Networks 3 Transport Layer TAMU Computer Networks 2 Application Layer TAMU Computer Networks 1 Introduction Overview in distributed systems and cloud computing 1 A well-optimized Union-Find implementation, in Java A heap implementation supporting deletion TAMU Advanced Algorithms 3, Maximum Bandwidth Path (Dijkstra, MST, Linear) TAMU Advanced Algorithms 2, B+ tree and Segment Intersection TAMU Advanced Algorithms 1, BST, 2-3 Tree and Heap TAMU AI, Searching problems Factorization Machine and Field-aware Factorization Machine for CTR prediction TAMU Neural Network 10 Information-Theoretic Models TAMU Neural Network 9 Principal Component Analysis TAMU Neural Network 8 Neurodynamics TAMU Neural Network 7 Self-Organizing Maps TAMU Neural Network 6 Deep Learning Overview TAMU Neural Network 5 Radial-Basis Function Networks TAMU Neural Network 4 Multi-Layer Perceptrons TAMU Neural Network 3 Single-Layer Perceptrons Princeton Algorithms P1W6 Hash Tables & Symbol Table Applications Stanford ML 11 Application Example Photo OCR Stanford ML 10 Large Scale Machine Learning Stanford ML 9 Anomaly Detection and Recommender Systems Stanford ML 8 Clustering & Principal Component Analysis Princeton Algorithms P1W5 Balanced Search Trees TAMU Neural Network 2 Learning Processes TAMU Neural Network 1 Introduction Stanford ML 7 Support Vector Machine Stanford ML 6 Evaluate Algorithms Princeton Algorithms P1W4 Priority Queues and Symbol Tables Stanford ML 5 Neural Networks Learning Princeton Algorithms P1W3 Mergesort and Quicksort Stanford ML 4 Neural Networks Basics Princeton Algorithms P1W2 Stack and Queue, Basic Sorts Stanford ML 3 Classification Problems Stanford ML 2 Multivariate Regression and Normal Equation Princeton Algorithms P1W1 Union and Find Stanford ML 1 Introduction and Parameter Learning

Deal with zip file in python

2016-08-10

Test for ZIP file

Use zipfile.is_zipfile

import zipfile
print(zipfile.is_zipfile('samples/archive.zip'))
True

Read ZIP information

ZipFile can manipulte ZIP directly, for read, write or update.

Get file information

List all files archived in zip file. Use namelist and infolist method and get list of filenames or list of ZipInfo instances.

import zipfile
zf = zipfile.ZipFile('samples/archive.zip','r')
# list filenames
for name in zf.namelist():
    print name,
a.txt b.txt c.txt
# list file infomation
for info in zf.infolist():
    print info.filename, info.date_time, info.file_size
a.txt (2016, 7, 15, 0, 5, 58) 2
b.txt (2016, 7, 15, 0, 6, 8) 2
c.txt (2016, 7, 15, 0, 6, 14) 2

Retrive data read()

Use read(), fed it with filename as param, return as a string.

zf = zipfile.ZipFile('samples/archive.zip')
for filename in zf.namelist():
    # can use other than namelist,['there.txt','notthere.txt']
    try:
        data = zf.read(filename) # extract use read()
    except KeyError:
        print "Error: Did not find %s in zip file" % filename
    else:
        print filename, ':',
        print repr(data)
a.txt : 'a\n'
b.txt : 'b\n'
c.txt : 'c\n'

Unzip file extract()

zf = zipfile.ZipFile('samples/archive.zip','r')
# Extract a member from the archive to the current working directory
zf.extract('a.txt') # you may want to specify path param

# Extract all members from the archive to the current working directory
zf.extractall() # you may want to specify path param

Compress data

Create new zip file. Create a new ZipFile, use w mode and use write() method if you want to add file.

print('creating archive')
zf = zipfile.ZipFile('zipfile_write.zip',mode='w')
try:
    print('adding readme.txt')
    zf.write('readme.txt')
finally:
    print('closing')
    zf.close() # remember to close
creating archive
adding readme.txt
closing

But by default, it just wrap all files together, not compressing it. If you want compress data, use zlib module. And you can change the default compression mode from zipfile.ZIP_STORED to zipfile.ZIP_DEFLATED.

# try to change compression type
try:
    import zlib
    compression = zipfile.ZIP_DEFLATED
except:
    compression = zipfile.ZIP_STORED

modes = {zipfile.ZIP_DEFLATED: 'deflated', zipfile.ZIP_STORED: 'stored'}
print('creating archive')
zf = zipfile.ZipFile('zipfile_write_compression.zip',mode='w')
try:
    print('adding README.txt with compression mode'+ str(modes[compression]))
    zf.write('readme.txt',compress_type=compression)
finally:
    print('closing')
    zf.close()
creating archive
adding README.txt with compression modedeflated
closing

Creative Commons License
Melon blog is created by melonskin. This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
© 2016-2024. All rights reserved by melonskin. Powered by Jekyll.