Public speaking course notes Read "Dynamo, Amazon’s Highly Available Key-value Store" Read "Bigtable, A Distributed Storage System for Structured Data" Read "Streaming Systems" 3, Watermarks Read "Streaming Systems" 1&2, Streaming 101 Read "F1, a distributed SQL database that scales" Read "Zanzibar, Google’s Consistent, Global Authorization System" Read "Spanner, Google's Globally-Distributed Database" Read "Designing Data-intensive applications" 12, The Future of Data Systems IOS development with Swift Read "Designing Data-intensive applications" 10&11, Batch and Stream Processing Read "Designing Data-intensive applications" 9, Consistency and Consensus Read "Designing Data-intensive applications" 8, Distributed System Troubles Read "Designing Data-intensive applications" 7, Transactions Read "Designing Data-intensive applications" 6, Partitioning Read "Designing Data-intensive applications" 5, Replication Read "Designing Data-intensive applications" 3&4, Storage, Retrieval, Encoding Read "Designing Data-intensive applications" 1&2, Foundation of Data Systems Three cases of binary search TAMU Operating System 2 Memory Management TAMU Operating System 1 Introduction Overview in cloud computing 2 TAMU Operating System 7 Virtualization TAMU Operating System 6 File System TAMU Operating System 5 I/O and Disk Management TAMU Operating System 4 Synchronization TAMU Operating System 3 Concurrency and Threading TAMU Computer Networks 5 Data Link Layer TAMU Computer Networks 4 Network Layer TAMU Computer Networks 3 Transport Layer TAMU Computer Networks 2 Application Layer TAMU Computer Networks 1 Introduction Overview in distributed systems and cloud computing 1 A well-optimized Union-Find implementation, in Java A heap implementation supporting deletion TAMU Advanced Algorithms 3, Maximum Bandwidth Path (Dijkstra, MST, Linear) TAMU Advanced Algorithms 2, B+ tree and Segment Intersection TAMU Advanced Algorithms 1, BST, 2-3 Tree and Heap TAMU AI, Searching problems Factorization Machine and Field-aware Factorization Machine for CTR prediction TAMU Neural Network 10 Information-Theoretic Models TAMU Neural Network 9 Principal Component Analysis TAMU Neural Network 8 Neurodynamics TAMU Neural Network 7 Self-Organizing Maps TAMU Neural Network 6 Deep Learning Overview TAMU Neural Network 5 Radial-Basis Function Networks TAMU Neural Network 4 Multi-Layer Perceptrons TAMU Neural Network 3 Single-Layer Perceptrons Princeton Algorithms P1W6 Hash Tables & Symbol Table Applications Stanford ML 11 Application Example Photo OCR Stanford ML 10 Large Scale Machine Learning Stanford ML 9 Anomaly Detection and Recommender Systems Stanford ML 8 Clustering & Principal Component Analysis Princeton Algorithms P1W5 Balanced Search Trees TAMU Neural Network 2 Learning Processes TAMU Neural Network 1 Introduction Stanford ML 7 Support Vector Machine Stanford ML 6 Evaluate Algorithms Princeton Algorithms P1W4 Priority Queues and Symbol Tables Stanford ML 5 Neural Networks Learning Princeton Algorithms P1W3 Mergesort and Quicksort Stanford ML 4 Neural Networks Basics Princeton Algorithms P1W2 Stack and Queue, Basic Sorts Stanford ML 3 Classification Problems Stanford ML 2 Multivariate Regression and Normal Equation Princeton Algorithms P1W1 Union and Find Stanford ML 1 Introduction and Parameter Learning

Basic linux commands

2016-08-20

Some very basic linux commands for quick reference.

  • ls: list files
    • ls, basic list
    • ls -l, one per line, long format
    • ls -a, list all files, including hidden file
    • ls -la, combination of the two
    • ls -lh, with file size in human readable format
    • ls -lS, long format list sorted by size (descending)
    • ls -lt, sort by modification time, newest first
    • ls -ltr, sort by time in reverse order
  • less: open file and read by page
    • less source_file, open a file and onle show the first page, less
    • d to next page; p to previous page
    • g to start; G to end
    • /kw for forward search; ?kw for backword search; n next and N previous
    • q to quit
  • cp: copy file or folder
    • cp /path/to/origin /path/to/copy, copy file
    • cp -r /path/to/origin /path/to/copy, copy folder, recursive
    • cp -vr /path/to/origin /path/to/copy, verbose mode, show it
    • file name can be specified with wildcard
  • mv: move or rename files and directories
    • mv source target, move or rename
    • mv -v source target, verbose mode
    • mv -f source target, overwriting with out confirmation
  • rm: remove files or directories
    • rm /path/to/file, remove file
    • rm -r /path/to/folder, remove folder, recursively
    • rm -rf /path/to/folder, remove folder without prompt
    • rm -i /path/to/file, interactive mode, need confirmation
    • before use rm with wildcards, try it with ls first
  • wildcard: match filename
    • *, match any characters
    • ?, match any single character
    • [characters], match member character of set
      • [abc], match a, b or c
      • [:alnum:], match alphanumeric characters
      • [:alpha:], match alphabetic characters
      • [:digit:], match numerals
      • [:upper:], match uppercase alphabetic characters
      • [:lower:], match lowercase alphabetic characters
    • [!characters], match opposite of [characters]
  • I/O Redirection
    • stdout: redirect standard output to file
      • ls, by default to standard output, like display
      • ls > file_list.txt, use > to redirect, write mode
      • ls >> file_list.txt, use » to redirect, append mode
    • stdin: get standard input from file
      • sort < file_list.txt, use < as standard input
      • sort < file_list.txt > sort_file_list.txt, combine stdin and stdout
    • pipelines |: stdout of one cmd is fed into the stdin of another
      • ls -l | less, powerful pipeline |
      • ls -lt | head -5, dispalys 5 newest files
      • du | sort -nr, display list of dirs and space consumed, sort it, largest first
    • filter: take stdin, perfrom operation, send results to stdout
      • sort:
        • sort filename, sort a file in ascending order
        • sort -r filename, sort a file in descending order
        • sort -n filename, sort a file using numeric rather than alphabetic order
        • sort -u filename, sort and preserve only unique lines
      • uniq: return unique line, needs to be sorted first
        • sort file | uniq, display each line once
        • sort file | uniq -u, display only unique lines
        • sort file | uniq -d, display only duplicated lines
        • sort file | uniq -c, display unique lines with occurences
        • sort file | uniq -c | sort -nr
      • grep: examine lines of input, output lines that contain the pattern
        • grep query_string path/to/file, search for an eaxct string
        • grep -i query_string path/to/file, in case-insensitive mode
        • grep -rI query_string ., search current dir recursively, ignore binary file
        • grep -C 3 query_string path/to/file, context with 3 lines
        • grep -c query_string path/to/file, count of match instead of matching text
        • grep -n query_string path/to/file, line number for each match
        • grep -l query_string path/to/file, list of files with matches
        • cat path/to/file | grep query_string, search stdin using pipeline
      • head: output the first part of files
        • head -n count_of_lines filename, first n lines
        • head -c number_in_bytes filename, first n bytes
      • tail: output the last part of files
        • tail -n count_of_lines filename, last n lines
        • tail -c number_in_bytes filename, last n bytes
  • unzip: extract compressed files in a zip archive
    • unzip file, extract zip file
    • unzip compressed_file -d /path/to/put/extracted_file, extract zip file to folder
    • unzip -l file, list the contents of a zip file without extracting
    • zip, unzip, gzip, gunzip, tar
  • ssh: ssh user@your_ip
  • nohup: Allows for a process to live when the terminal gets killed
    • nohup command options
    • nohup command options &, run in the background
  • linux command

Creative Commons License
Melon blog is created by melonskin. This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
© 2016-2024. All rights reserved by melonskin. Powered by Jekyll.