When size breaks the usual tools.
WHAT BREAKS
Opening a file in an editor that loads it entirely Sorting something larger than memory Compressing without enough free space
WHAT TO USE INSTEAD OF AN EDITOR
less, which reads progressively.
WHAT TO USE FOR EXTRACTING PART
head and tail, or sed with a line range.
HOW TO TAKE A MIDDLE SECTION
Combine head and tail, or specify the range directly.
WHAT TO USE FOR COUNTING
wc, which streams.
WHAT TO BE CAREFUL WITH
Commands that buffer the whole file.
WHAT TO CHECK BEFORE COMPRESSING
Free space, against the original size.
WHY
Compression writes a new file before removing the old.
WHAT TO DO WHEN SPACE IS TIGHT
Compress to a different file system.
HOW TO SPLIT A LARGE FILE
The split command, by size or line count.
WHERE THAT HELPS
Transferring in pieces over an unreliable connection Importing a database dump too large to process at once
HOW TO REJOIN THEM
Concatenate in order.
WHAT TO DO WITH A LARGE DATABASE DUMP
Compress it, split it, and import with progress visible.
HOW TO SEE PROGRESS
Pipe through a tool that reports throughput.
WHY THAT MATTERS
An import with no output is indistinguishable from one that has hung.
