Tool - EOL Scanner
Author: Copyright (C) 2014 by CiderPress authors

EOL Scanner

EOL is an acronym for "end of line". Each line of a text file ends with an end-of-line marker. The Apple II and Macintosh use a carriage return, UNIX systems use a linefeed, and Windows uses a carriage return followed by a linefeed. These are usually abbreviated "CR", "LF", and "CRLF".

When text files are moved from one system to another, the end-of-line markers on text files need to be converted. Unfortunately, overzealous converters will sometimes convert a non-text file, resulting in corrupted data. The only way to tell if a file has been corrupted is to count up the occurrences of CR, LF, and CRLF, and see if they make sense.

The EOL Scanner tool does exactly that. The number of times CR, LF, and CRLF appear are counted and displayed. This information, combined with some knowledge of the format of the file, will tell you if a file has been corrupted by an EOL conversion. The tool also counts up "high-ASCII" characters to test for conversions to and from DOS text files.

Take for example a healthy 140K disk image. The scanner reports the following:

143360 characters

43582 high-ASCII characters

381 carriage returns

863 line feeds

3 CRLFs

A typical disk image will have a smattering of CR (hex value 0x0d) and LF (hex value 0x0a). Occasionally they will appear near each other and form a CRLF. A disk image with lots of text files will have many more CRs than LFs, while a disk with programs on it could have any amount of either. (To be accurate, a DOS 3.3 disk with text files won't show a large number of CRs, because DOS 3.3 uses "high ASCII" 0x8d instead of 0x0d.) Note that an occurrence of "CRLF" only updates the CRLF counter, not the CR and LF counters.

Now lets look at a disk image that doesn't seem to work. The scanner comes back with:

143360 characters

56085 high-ASCII characters

530 carriage returns

0 line feeds

0 CRLFs

The disk image has absolutely no line feeds whatsoever. A blank formatted ProDOS disk will have 3 or 4 carriage returns and line feeds, a blank unbootable DOS disk 1 of each. A non-empty disk should never have zero CRs or LFs. This disk was corrupted by a converter that changed all of the linefeeds to carriage returns.

Similar results hold for compressed data archives. Data that is well compressed will show a fairly even distribution of all possible characters, so a total absence of CR or LF is a big red flag. Of course, if the archive is only a few hundred bytes long, it's quite possible that no CRs or LFs will be found.

Disk images in nibble format (.nib) have no CRs or LFs in them. This is normal. The entire file should be "high ASCII" characters.

Besides its use as a diagnostic tool, the EOL Scanner can also tell you what format a text file is in, and also how many lines it has.