Whitespace, line endings and other invisible changes that pollute diffs

Unix-descended systems end a line with a single line feed, U+000A. Windows text files traditionally end it with a carriage return followed by a line feed, U+000D U+000A. Both are correct; they are simply different bytes. A file converted from one convention to the other differs on every single line, and a byte-oriented diff will say exactly that — thousands of changed lines with no visible change in any of them.

Git's core.autocrlf setting is the usual source of surprise. Set to true it converts CRLF to LF when you commit and back to CRLF when you check out; set to input it converts on commit only. When one developer has it on and another does not, files start flipping back and forth, and each flip is a whole-file diff. The durable fix is not a per-developer setting at all: commit a .gitattributes file with * text=auto so the repository decides, and add explicit rules such as *.sh text eol=lf and *.bat text eol=crlf for files that must keep one form.

To see what you actually have, run file on the path — it reports “with CRLF line terminators” — or open the file with cat -A on Linux, where a carriage return shows up as ^M. DiffCanvas detects the line-ending style of each side and tells you when they differ, and it never reports a CRLF-versus-LF difference as a changed line, because that would hide every real change underneath it.

Trailing whitespace is invisible by definition: one or more spaces or tabs sitting after the last visible character. Editors add it when you leave the cursor at the end of an empty indented line; formatters remove it. Neither changes what the code does, but both produce changed lines, and a commit that mixes a real fix with a trailing-space sweep is hard to review honestly.

Indentation style is the same problem at a larger scale. Converting a file from tabs to spaces, or from four-space to two-space indentation, rewrites the leading whitespace of nearly every line. Both git and every serious diff viewer can ignore it on demand: git diff -w ignores all whitespace, -b ignores changes in the amount of whitespace, and --ignore-blank-lines skips lines that only became or stopped being empty. DiffCanvas offers the same choices separately, so you can ignore trailing whitespace while still seeing an indentation change that actually matters, for example inside a YAML file where indentation is structure.

The habit worth building is to keep formatting changes in their own commit. A pure whitespace commit is boring to review and easy to verify with git diff -w, which should print nothing at all; a mixed commit forces every reviewer to do that separation by hand.

A byte-order mark is U+FEFF at the very start of a file, stored as the three bytes EF BB BF in UTF-8. It carries no text and exists only to announce the encoding, but it is part of the file, so a BOM added by a Windows editor makes the first line differ and can break shell scripts, JSON parsers and PHP files that must emit nothing before their output.

Encoding changes are worse, because they can alter every byte that is not plain ASCII. A file saved as UTF-8 and then re-saved as Windows-1251 or Latin-1 keeps the same letters on screen in the editor that did it and becomes a different file everywhere else. If a diff shows changes only on lines containing accented letters, Cyrillic or emoji, check the encoding before you look for anything cleverer.

DiffCanvas ignores a leading BOM when it compares, so it can never be the reason a first line is marked as changed, but it does tell you when only one of the two sides has one — that is usually the signal you actually needed.

Unicode contains many pairs of characters that render almost identically. A non-breaking space, U+00A0, is indistinguishable from an ordinary space in most fonts and is inserted by word processors, some web copy-paste and a few keyboard layouts. A zero-width space, U+200B, has no width at all. Curly quotes and apostrophes substituted by autocorrect look right in prose and break string literals in code.

Alphabet confusion is the sharpest version of this. Cyrillic а, е, о, р, с and х are visually identical to Latin a, e, o, p, c and x in most typefaces, so a hostname or an identifier can be wrong in a way that no amount of staring will reveal. A diff will mark the line as changed and the word-level highlight will point at the exact character, which is often the fastest way to find it.

When you suspect a look-alike, stop looking and start measuring. Pipe the line through a hex dump, or paste both versions into DiffCanvas and turn on word-level marks: the highlight lands on a single character, and copying that character into a search tells you what it is.

An .editorconfig file at the root of a project is the cheapest prevention there is. Editors and IDEs read it without a plugin in most cases, and a handful of lines — indent_style, indent_size, end_of_line = lf, charset = utf-8, trim_trailing_whitespace = true, insert_final_newline = true — remove four of the five problems in this article before they reach a commit.

Pair it with .gitattributes for the rules git itself enforces, and with a formatter run by a pre-commit hook for the rest. Git can also warn you: git diff --check reports trailing whitespace and space-before-tab in what you are about to commit, and core.whitespace lets you choose which of those count as errors.

None of this helps with a file someone has already sent you, and that is where the ignore options earn their place. Compare as-is first, so you can see whether the invisible difference exists at all; then switch trailing or leading whitespace off and watch how many changed lines survive. The number that remains is your real diff.