Lots of changes.HEADmaster

3 files changed, 270 insertions, 90 deletions

diff --git a/README b/README
index b5ce2b3..8088f3e 100644
--- a/README
+++ b/README
@@ -32,14 +32,19 @@ Requirements:
   https://github.com/espressif/esptool
   ...or your OS may provide a package for this. I'm not sure
   what the minimum version is. I've tested this with 4.4, which
-  is pretty old (November 2021). If you have trouble with other
-  versions, please let me know.
+  is pretty old (November 2021), and 4.8 which is less old (September
+  2024). If you have trouble with other versions, please let me know.
 
-- pio (from PlatformIO), for viewing the FujiNet's debug log after
-  it's been flashed (this is optional but highly recommended). See:
+- pio (from PlatformIO), for autodetecting the FujiNet and viewing
+  the FujiNet's debug log after it's been flashed (this is optional
+  but highly recommended). See:
   https://platformio.org/
   ...or your OS may provide a package for this.
 
+- sha256sum, for verifying the downloaded firmware .zip files. On
+  Linux, this is generally provided by the coreutils package, so you
+  probably already have it installed.
+
 Installation:
 
 It's not really necessary (fujinet-updater can run from wherever
diff --git a/TODO b/TODO
index 55b57f6..6af7bfe 100644
--- a/TODO
+++ b/TODO
@@ -15,35 +15,20 @@ Stuff to do before a 1.0 release:
 - Test with older platformio versions.
 
 - Find out if it works on MacOS as-is (if not, make it work).
-  I think the device nodes aren't /dev/ttyUSB* there... though
-  the -p option should work, for users who know what they're doing.
+  I think the device nodes aren't /dev/ttyUSB* there... though pio
+  should be smart enough to know them, and the -p option should work,
+  for users who know what they're doing.
 
 - Config file in ~/.fujinet-updater.conf, to set at least the default
   -t option. Don't actually put this in ~/.fujinet-updater/, since
   the user might want to rm -rf that to clean out old firmwares.
 
-- See if we can autodetect which /dev/ttyUSB* is the FujiNet.
-  Maybe "esptool.py chip_id" or such. If we detect multiple
-  FujiNets (or at least multiple ESP32 devices), show a list
-  & let the user choose (unless -f). It looks like we can
-  actually read 256 bytes or so of the FujiNet's flash (at offset
-  0x10000) and check for the string FujiNet. We can even check the
-  bytes we read against what we're about to flash, and tell the user
-  the operation is unnecessary!
-
-- -c option to force the use of curl instead of wget, if both
-  are installed.
-
 - -d option to delete the ~/.fujinet-updater/ dir afterwards?
   or maybe make this the default, and use -k to keep it? Or
   maybe only delete the downloaded/extracted files, because...
 
 - ...we might log stdout/stderr to ~/.fujinet-updater/log.txt
-  or simiar.
-
-- The github releases json file has sha hashes, it would be nice
-  to check those after downloading a file... or *not* downloading
-  the file if we find an existing copy.
+  or similar.
 
 - -v option for verify? easy to implement (esptool.py has an option
   for it already).
diff --git a/fujinet-updater b/fujinet-updater
index d301f00..c4bffce 100755
--- a/fujinet-updater
+++ b/fujinet-updater
@@ -4,7 +4,9 @@
 use JSON;
 use Getopt::Std;
 
-##use Data::Dump 'dump';
+# so is this one, but becase it's optional, proceed without it if
+# it's missing.
+eval { require Digest::SHA; Digest::SHA->import('sha256_hex'); $sha_ok = 1 };
 
 $VERSION="0.0.1";
 
@@ -25,7 +27,7 @@ sub slurpfile {
 # no arguments. so this is simpler than expected.
 sub checkpath {
 	my $exe = shift;
-	my $result = system("$exe 1>/dev/null 2>&1");
+	my $result = system("$exe </dev/null 1>/dev/null 2>&1");
 	$result >>= 8;
 	if($result < 0 || $result > 2) {
 		warn "can't find $exe on PATH.\n";
@@ -36,12 +38,22 @@ sub checkpath {
 }
 
 sub check_paths {
-	if(checkpath("wget")) {
-		# wget follows redirects by default.
-		$dlcmd = "wget -O";
+	# the -L is needed to make curl follow redirects.
+	my $curlcmd = "curl -L -o";
+
+	# wget follows redirects by default.
+	my $wgetcmd = "wget -nv -O";
+
+	if($opts{c}) {
+		if(checkpath("curl")) {
+			$dlcmd = $curlcmd;
+		} else {
+			die_path("curl");
+		}
+	} elsif(checkpath("wget")) {
+		$dlcmd = $wgetcmd;
 	} elsif(checkpath("curl")) {
-		# the -L is needed to make curl follow redirects.
-		$dlcmd = "curl -L -o";
+		$dlcmd = $curlcmd;
 	} else {
 		die_path("curl or wget");
 	}
@@ -52,25 +64,81 @@ sub check_paths {
 
 	if(!checkpath("pio")) {
 		if(!$opts{n}) {
-			warn "can't find pio on PATH, disabling monitoring.\n";
+			warn "  disabling monitoring.\n";
 			$opts{n} = 1;
 		}
+		if(!$opts{p}) {
+			die "  can't autodetect FujiNet, install pio or use -p option.\n";
+		}
+	}
+}
+
+sub prepend_sudo {
+	my @ret;
+	for my $cmd (@_) {
+		$cmd = "sudo " . $cmd if $opts{s};
+		push @ret, $cmd;
+	}
+	return @ret;
+}
+
+sub check_digest {
+	if(!$sha_ok) {
+		warn "can't verify digest because Digest::SHA module is missing.\n";
+		return;
+	}
+
+	my $file = shift;
+	my $cksum = shift;
+
+	if(!$cksum) {
+		warn "No digest provided for this file.\n";
+		return;
+	}
+
+	my($algo, $digest) = split /:/, $cksum;
+
+	if($algo ne 'sha256') {
+		warn "don't know how to check $algo checksums, sorry.\n";
+		return;
+	}
+
+	my $content = slurpfile($file);
+	my $res = sha256_hex($content);
+	if($digest eq $res) {
+		print "\n$algo digest verified.\n";
+	} else {
+		print "\n$algo digest does not match.\n";
+		print "Use file anyway [y/N]? ";
+		chomp(my $yn = <>);
+		if($yn !~ /y/i) {
+			print "Delete it? [Y/n]? ";
+			chomp($yn = <>);
+			if($yn !~ /n/i) {
+				unlink $file;
+			}
+			exit(1);
+		}
 	}
 }
 
 sub download_firmware {
 	my $url = shift;
+	my $cksum = shift;
 	my $file = $url;
 	$file =~ s,.*/,,;
 
 	if(-e $file) {
 		print "$file already exists, download anyway [y/N]? ";
 		chomp(my $c = <>);
-		return $file if $c !~ /^y/i;
+		if($c !~ /^y/i) {
+			check_digest($file, $cksum);
+			return $file;
+		}
 	}
 
 	system("$dlcmd $file $url");
-	# TODO: check for errors in download
+	# TODO: check for errors from wget/curl? we already check the file thoroughly...
 
 	if(! -e $file) {
 		die "failed to download $url!\n";
@@ -84,10 +152,20 @@ sub download_firmware {
 		die "$file is not a .zip file!\n";
 	}
 
+	check_digest($file, $cksum);
+
 	return $file;
 }
 
 sub show_choices {
+	if(!$opts{s}) {
+		print "\nFujiNet platform [$sys]? ";
+		my $p = <>;
+		chomp $p;
+		$sys = uc $p if length($p);
+	}
+
+	print "\nChecking for $sys releases.\n\n";
 	my $tmpfile = "releases.$$.json";
 	unlink $tmpfile;
 	system("$dlcmd releases.$$.json $RELEASE_URL");
@@ -103,23 +181,22 @@ sub show_choices {
 	unlink $tmpfile;
 
 
-	##dump($j);
-	#for(@{$j}) {
-		#print "$_  " for keys %{$_};
-	#}
-	#die;
-
 	my $num = 1;
 	my @choices;
+	my %checksums;
+
+	print "\nChoose release:\n";
 
 	for(@{$j}) {
 		for(@{$_->{assets}}) {
 			my $url = $_->{browser_download_url};
+			my $d = $_->{digest};
 			next unless $url =~ /\/fujinet-[A-Z].*\.zip$/;
 			if($sys ne "ALL") {
 				next unless $url =~ /-$sys-.*\.zip$/i;
 				next if $sys eq "ATARI" && $url =~ /-8mb/;
 			}
+			$checksums{$url} = $d;
 			my $file = $url;
 			$file =~ s,.*/,,;
 			$choices[$num] = $url;
@@ -127,11 +204,18 @@ sub show_choices {
 			$num++;
 		}
 	}
+
+	if(!@choices) {
+		die "No firmware releases for $sys!\n";
+	}
+
 	print "-> ";
 	chomp(my $c = <>);
 	die "invalid choice.\n" unless $choices[$c];
 
-	return $choices[$c];
+	my $url = $choices[$c];
+	my $cksum = $checksums{$url};
+	return($url, $cksum);
 }
 
 sub get_esptool_cmds {
@@ -141,7 +225,9 @@ sub get_esptool_cmds {
 	}
 	if(-d $file) {
 		$dir = $file;
+		print "Using extracted dir $dir\n";
 	} else {
+		print "Extracting $file\n";
 		$dir = $file;
 		$dir =~ s,\.zip$,,;
 		$dir =~ s,.*/,,;
@@ -156,37 +242,108 @@ sub get_esptool_cmds {
 	my $json = slurpfile("release.json");
 	my $j = decode_json($json) or die "$!";
 
-	my $sudo = $opts{s} ? "sudo " : "";
 	for(@{$j->{files}}) {
 		my $f = $_->{filename};
 		die "$f referenced in release.json, but doesn't exist.\n"
 			unless -e $f;
-		push @commands, $sudo .
+		push @commands,
 			  "esptool.py --port $port --baud $baud " .
 				  "write_flash $_->{offset} $f";
 	}
 
+	if($mon) {
+		push @commands, "pio device monitor -b $baud -p $port";
+	}
+
+	@commands = prepend_sudo(@commands);
+
 	die "no filenames in release.json\n" unless @commands;
 
 	return @commands;
 }
 
 sub confirm {
-	print "About to run these commands to flash the firmware:\n";
+	print "\nAbout to run these commands to flash the firmware:\n";
 	print "   $_\n" for @commands;
-	print "Are you sure you want to proceed[y/N]? ";
+	print "\nAre you sure you want to proceed[y/N]? ";
 	chomp(my $resp = <>);
 	return $resp =~ /^y/i;
 }
 
+sub port_menu {
+	my $choice = 1;
+	my @p;
+	print "\nFound multiple devices.\n";
+	for(@_) {
+		print "[$choice] " . $_ . "\n";
+		$p[$choice++] = $_;
+	}
+	print "-> ";
+	chomp($choice = <>);
+	my $port = $p[$choice];
+	die "Invalid choice\n" unless $port;
+	return $port;
+}
+
+# TODO: be a little more specific. Right now it just assumes if pio
+# doesn't call the hardware "n/a" that it's really a FujiNet.
+# Both my FujiNets are VID:PID=10C4:EA60, CP2102N USB to UART Bridge Controller,
+# but I don't know if *all* FujiNets will identify as this.
+sub detect_fujinet {
+	my @cmd = ("pio --no-ansi device list");
+	@cmd = prepend_sudo(@cmd);
+	open my $fh, '-|', $cmd[0] or die "pio failed\n";
+
+	print "\nDetecting FujiNet(s)...\n";
+
+	my $tty;
+	my @res;
+	my $vidpid;
+	while(<$fh>) {
+		chomp;
+		if(m,(/dev/\S+),) {
+			$tty = $1;
+			next;
+		}
+		if(m,^Hardware ID: USB (VID:PID=\S+),) {
+			$vidpid = $1;
+			push @res, $tty;
+			next;
+		}
+		if(!m,n/a, && m,^Description: (\S+),) {
+			print "$tty: $vidpid $1\n";
+			next;
+		}
+	}
+	close $fh;
+
+	if(!@res) {
+		print "\npio couldn't find a FujiNet, are you sure it's plugged in?\n";
+		print "If you're sure you know what you're doing, re-run with -p <port> option.\n";
+		exit(1);
+	} elsif(@res == 1) {
+		$port = $res[0];
+		print "\nFound $port, using.\n";
+	} else {
+		$port = port_menu(@res);
+		print "\nOK, using $port\n";
+	}
+
+	return $port;
+}
+
 sub interact {
 	mkdir($homedir);
 	chdir($homedir) or die "$homedir: $!\n";
 
 	check_paths();
 
-	$url = show_choices();
-	$file = download_firmware($url);
+	$port = detect_fujinet() unless $port;
+	if(! -w $port) {
+		print "\nCurrent user has no write permission for $port! Try -s option.\n";
+	}
+	($url, $cksum) = show_choices();
+	$file = download_firmware($url, $cksum);
 
 	@commands = get_esptool_cmds($file);
 
@@ -203,11 +360,6 @@ sub flash {
 		print "===> $_\n";
 		system($_);
 	}
-
-	if($mon) {
-		my $sudo = $opts{s} ? "sudo " : "";
-		system($sudo . "pio device monitor -b $baud -p $port");
-	}
 }
 
 ### main()
@@ -226,18 +378,26 @@ if(@ARGV) {
 	}
 }
 
-getopts('st:b:p:nf:', \%opts);
+getopts('st:b:p:nc', \%opts) or die "bad option\n";
+
+if(@ARGV) {
+	$opts{f} = shift @ARGV;
+	if(@ARGV) {
+		die "only one file/directory is supported.\n";
+	}
+}
 
 # TODO: option to override this?
 $homedir = "$ENV{HOME}/.fujinet-updater";
 
 $baud = $opts{b} || 460800;
-$port = $opts{p} || "/dev/ttyUSB0";
+$port = $opts{p};
 $sys  = uc($opts{t} || "atari");
 $mon  = !$opts{n};
 
 if($opts{f}) {
 	$file = $opts{f};
+	$port = $opts{p} || "/dev/ttyUSB0";
 	if($file !~ /^\//) {
 		my $p = `pwd`;
 		chomp($p);
@@ -264,7 +424,7 @@ fujinet-updater - download and flash the firmware on a FujiNet device.
 
 =head1 SYNOPSIS
 
-B<fujinet-updater> [B<-s>] [B<-t> I<board-type>] [B<-b> I<baud>] [B<-p> I<serial-port>] [B<-n>] [B<-f> I<zipfile-or-directory>]
+B<fujinet-updater> [B<-s>] [B<-t> I<board-type>] [B<-b> I<baud>] [B<-p> I<serial-port>] [B<-n>] <I<zipfile-or-directory>>
 
 =head1 DESCRIPTION
 
@@ -275,15 +435,17 @@ and uses B<esptool.py> to upload the firmware to the device and
 (if installed) B<pio> to monitor the FujiNet's log messages when it
 reboots after being flashed.
 
-Normally, B<fujinet-updater> runs interactively. It searches for
-firmware releases for the target system, and presents a menu from
-which you can choose the version to flash. After downloading and
-unzipping the firmware, you will be show the exact B<esptool.py>
-commands that will be executed, and offered a last chance to abort
-before flashing the device.
+Normally, B<fujinet-updater> runs interactively. It attempts to
+autodetect the FujiNet's USB serial device, and prompts for the
+target system. Then it searches for firmware releases for the target
+system, and presents a menu from which you can choose the version to
+flash. After downloading and unzipping the firmware, you will be shown
+the exact B<esptool.py> commands that will be executed, and offered a
+last chance to abort before flashing the device.
 
-For non-interactive use, use the B<-f> option to choose an
-already-downloaded (and possibly already-extracted) firmware zip file.
+For non-interactive use, you can give the path to an
+already-downloaded (and possibly already-extracted) firmware zip file
+or directory.
 
 =head1 OPTIONS
 
@@ -298,52 +460,76 @@ command.
 
 Prints this documentation in man page (troff/nroff source) format.
 
+=item B<--version>
+
+Prints the B<fujinet-updated> version and exits.
+
 =item B<-s>
 
-Run B<esptool.py> commands with B<sudo>(8). If you haven't already
-authenticated with sudo in the current session, you'll be asked for
-your password when the first command is run.
+Run B<pio> and B<esptool.py> commands with B<sudo>(8). If you haven't
+already authenticated with sudo in the current session, you'll be
+asked for your password when the first command is run.
+
+It's better to avoid this option, if possible. Example: on Slackware
+Linux, the B</dev/ttyUSB*> devices are owned by user I<root>, group
+I<dialout>, and the group has read/write permission. So, you can add
+your user to the I<dialout> group (using e.g. B<usermod>(8)), and
+you'll be able to access the devices without using B<sudo>. Note
+that if you add your user to a new group, you generally have to
+log out and back in for the new group to be usable.
+
+=item B<-c>
+
+Use B<curl>, if both B<curl> and B<wget> are installed. Normally,
+B<wget> is the default. There's no particular reason to prefer
+B<curl> or B<wget> (both work fine).
 
 =item B<-t> I<board-type>
 
-Which system (platform) your FujiNet board is designed for. This is
-case-insensitive. Default is I<atari>. Other choices include I<atari-8mb>,
-I<adam>, I<apple>, I<coco>, etc. Use B<-t> I<all> to see all platforms.
+Which system (platform) your FujiNet board is designed for. This
+is case-insensitive. Default is I<atari>. Other choices include
+I<atari-8mb>, I<adam>, I<apple>, I<coco>, etc. Use B<-t> I<all> to
+see all platforms. In interactive mode, you are prompted for the
+platform, and this just sets the default.
 
-I<NOTE>: Only I<atari> has been tested!
+B<NOTE>: Only I<atari> has been tested!
 
 =item B<-b> I<baud>
 
 Baud (or actually, bits-per-second) rate for the flashing process. Defaults
 to I<460800>. Normally you should not need to change this.
 
-=item B<-b> I<serial-port>
+=item B<-p> I<serial-port>
 
-Defaults to I</dev/ttyUSB0>. If you have multiple USB devices that appear as
-serial ports (including multiple FujiNets), you may need to change this to
-e.g. I</dev/ttyUSB1>.
+Serial port device for the FujiNet. In interactive mode (no filename
+argument), the default is to auto-detect (using B<pio device
+list>). In non-interactive mode, defaults to I</dev/ttyUSB0>.
 
 =item B<-n>
 
-Do not run B<pio> to monitor the debug output from the FujiNet when it reboots
-after being flashed. Normally you would want to see this output.
+Do not run B<pio device monitor> to monitor the debug output from the
+FujiNet when it reboots after being flashed. Normally you would want
+to see this output.
 
-=item B<-f> I<zipfile-or-directory>
+=item I<zipfile-or-directory>
 
 Use a specific firmware that's already been downloaded, rather than
 prompting for a file to download. With this option, B<fujinet-updater>
 does not prompt the user for anything, I<including> the final
-confirmation prompt before flashing the device.
+confirmation prompt before flashing the device. It also doesn't
+autodetect the FujiNet (use the B<-p> option, if the port isn't
+I</dev/ttyUSB0>).
 
-If the argument to B<-f> is a zip file, it will be extracted to a
-temporary directory. Either way, the zip file or directory B<must>
-contain the B<release.json> file that lists the firmware files and
-their offsets, and all files mentioned there B<must> be present.
+If the argument is a directory, it will be used as-is. If it's a zip
+file, it will be extracted to a temporary directory. Either way, the
+zip file or directory B<must> contain the B<release.json> file that
+lists the firmware files and their offsets, and all files mentioned
+there B<must> be present.
 
-When using B<-f>, no network activity is done (so this can also be
-considered "offline" mode).
+When using a local file, no network activity is done (so this can also
+be considered "offline" mode).
 
-For I<totally> non-interactive mode, you'll want to combine B<-f>
+For I<totally> non-interactive mode, you'll want to combine this
 with B<-n> (to disable monitoring).
 
 =back
@@ -358,14 +544,18 @@ The official fujinet-flasher is a Python3 GUI program. Python3 is
 still a fast-moving target, and its developers don't care about
 supporting older versions of Python3 (or older Linux glibc versions,
 for their binary releases). Plus, it's graphical. Some of us prefer
-command-line tools.
+command-line tools, especially ones that can be run non-interactively.
 
 But, fujinet_firmware_uploader.py is a command line tool, isn't
-it? Yes, but as shipped, it's broken: hardcoded weird place where it
-looks for the esptool.py executable, and it only flashes 2 of the files
-in the release zip file (no idea why, but this leaves the FujiNet in
-an unusable state). Also, it always downloads and extracts its files
-to the current directory, which can be annoying.
+it? Yes, but as shipped, it's broken: hardcoded weird place where
+it looks for the esptool.py executable, and it only flashes 2 of
+the files in the release zip file (no idea why, but this leaves the
+FujiNet in an unusable state). Also, it always downloads I<all> the
+firmware .zip files it finds, and extracts its files to the current
+directory, which can be annoying... and if you need to use B<sudo>
+to read/write to the serial port, root access is also used to download the
+files. B<fujinet-updater> doesn't do anything as root except actually
+communicate with the serial port.
 
 B<fujinet-updater> can be used non-interactively (e.g. from a Makefile
 or other script), which is not true of either of the existing tools.