Q: What is sbosrcarch? A: sbosrcarch is "The SlackBuilds.Org Source Archive". It contains copies of the source files listed in the .info files for all (or almost all) the builds on SlackBuilds.org. sbosrcarch is also the name of the software that created and maintains the archive (more about this later, near the end of this FAQ). Q: What is sbosrcarch for? A: It's intended to be a backup location for source files that can't be downloaded. This happens mainly for these reasons: - The upstream web site goes down, is moved, or has connectivity issues (intermittent or long-term). - Upstream moves or removes the source, when they release a new version. Also, the archive is hosted on a fast, well-connected host. Sometimes you might choose to use the archive just for faster downloads. A side benefit of the archiving process is that the archive maintenance software produces a log of failed downloads, which can then be sent to the slackbuilds-users mailing list and/or build maintainer so it can be fixed quickly. Q: Who is responsible for sbosrcarch? A: The archive server is operated by Darren Austin, aka "Tadgy" on Freenode IRC. The archive script was written by B. Watson, aka "Urchlay" on Freenode. Both of us keep an eye on the logs and keep the archive healthy. The best way to contact us is using an IRC client to connect to Freenode and join the ##slackware or ##slackbuilds channel. We can also be reached by email: B. Watson TODO: make sure Tadgy is OK with his email being here! Please read this entire FAQ before asking us questions. Chances are, you'll find the answer here. If not, or if the answer isn't clear enough, we'll be happy to help. Note that the SlackBuilds.org team is NOT responsible for the archive. PLEASE don't bother them with questions about sbosrcarch, they're already busy enough maintaining the actual SlackBuilds site! Same goes for individual build maintainers. Q: Why create a giant archive like this? Isn't it better to fix the SlackBuilds whose sources can't be downloaded? A: Sort-of. Yes, if a SlackBuild references a no-longer-existing source download URL, it should be updated. Usually the SlackBuild maintainer is responsible for this. Sometimes the SBo admins take care of it instead. Sometimes, it takes longer than expected to update a SlackBuild: the new version uses a different build system, or requires some dependency to be updated first, or the maintainer is too busy with Real Life and can't spare the time just at the moment. Once the build is updated, it still doesn't appear instantly on the site. It has to sit in the "pending" queue until it's been reviewed by the admins, and then in the "ready" queue until the next public update. The SBo update process is complex, and requires coordination between the various admins. Generally this means that site updates ("Public www update" in the git log) only happen once a week. During the time it takes for the SlackBuild to get updated for the new download URL (and possibly new version), users won't be able to download the source as listed on the SBo site. That's what the archive is mainly intended for. It's a fallback, a stop-gap solution, that allows builds to keep working during the period between the source disappearing and the build being updated. Usually this is only a week or less, but sometimes things slip through the cracks... Q: How do I use the archive? A: Several answers here: - Using a tool that supports the archive, such as sbopkg or sbotools. This is by far the easiest way: they automatically use the archive if they need to, without you having to do any extra work. - Manually with a web browser. The easy way is to start at: http://slackware.uk/sbosrcarch/by-name/ ...which shows a list of category directories (academic, accessibility, audio, etc). Choose a category, then within the category you'll see a list of build name directories. Each of these will contain the source file(s) for the build. Example: you can't download the source to system/atari800, so you go to the by-name page, click on "system", then "atari800". There you'll see the file you wanted, atari800-3.1.0.tar.gz (unless it's been updated since I wrote this). - With a download tool like wget or curl. You could do this using the same by-name tree as you would for manual lookups, but it's better to do this by md5sum. The base URL for this is: http://slackware.uk/sbosrcarch/by-md5/ In the build's .info file, take the 'filename' part of each download URL. Example: "atari800-3.1.0.tar.gz", where the link is http://downloads.sourceforge.net/project/atari800/atari800/3.1.0/atari800-3.1.0.tar.gz Now take the MD5SUM (or MD5SUM_x86_64 if you're using DOWNLOAD_x86_64), and use the first two characters as subdirectory names, followed by the full md5sum. Example: we have MD5SUM="354f8756a7f33cf5b7a56377d1759e41" in the .info file. The directory for this would be: 3/5/354f8756a7f33cf5b7a56377d1759e41 Add this to the base URL and get: http://slackware.uk/sbosrcarch/by-md5/3/5/354f8756a7f33cf5b7a56377d1759e41/ Now add the filename part from DOWNLOAD or DOWNLOAD_x86_64, and you get: http://slackware.uk/sbosrcarch/by-md5/3/5/354f8756a7f33cf5b7a56377d1759e41/atari800-3.1.0.tar.g This is the exact URL for the file, if it's actually present in the archive. Most likely, it will be, and your download will succeed. If the download fails, the file's not in the archive. Of course, all these steps should be automated. You'll end up writing a script in your favorite language to do the job. Or: - Using the sbosrc script Same as above, except someone's already written it for you. Download it here: http://urchlay.naptime.net/repos/sbostuff/plain/sbosrc ...or, it'd be better to use git: git clone git://urchlay.naptime.net/sbostuff.git Make it executable (chmod +x) and place it somewhere on your $PATH, such as /usr/local/bin. Whenever you need to download something from the archive, change to the directory containing the .info file (same place as the .SlackBuild) and just run: sbosrc ...which will check the current architecture (32-bit or 64-bit), parse the info file, calculate the URL as above, and download the file to the current directory. Q: I need a specific older version of a source file, not the latest version that's packaged on SBo. Will the archive have it? A: Probably not. Old versions don't disappear immediately when new ones are archived, but they do get purged monthly... or, almost: old files are deleted on the 30th of every month, and February is only 28 or 29 days long! Use the by-md5 tree if you're looking for an old version, since some builds use unversioned filenames (new one will overwrite the old, in the by-name tree). If you know the exact filename and/or md5sum, you can always try a google search for them. Use "quotes" around the filename. Q: How do I know it's safe to use files downloaded from the archive? A: The same way you know it's safe to use any file you downloaded for use with a SlackBuild: check the downloaded file's md5sum against the MD5SUM line in the build's .info file. Q: How do I use the archive with automated tools such as sbopkg and sbotools? A: For sbopkg and sbotools, you just run them normally. They'll automatically search the archive, if a source download fails. Q: How complete is the archive? A: Currently (2018-06-26), the by-md5 tree is 100% complete. This does NOT count blacklisted sources (see next question). For a more up-to-date answer, see the archive status page: http://slackware.uk/sbosrcarch/STATUS This gets updated nightly. Q: Why are some sources missing from the archive? A: Multiple answers: - The archiver couldn't download the file. Maybe the site was down when it tried, or the upstream developers removed the file. Generally this will require the build's maintainer to fix the .info file or update the SlackBuild to a newer version (that actually exists). In some cases, the archive operator will find the file and manually add it to the archive. - The archiver downloaded the file, but the download's md5sum doesn't match. The build maintainer will have to fix the .info file. We won't archive any files we can't verify by md5sum. - There is some software that can't be automatically downloaded (requires account creation on the upstream site) or whose license doesn't allow us to redistribute it. The classic example of both is development/jdk: Oracle's license requires that users download the file directly from their site and doesn't allow us (or anyone else) to offer it for download. Also, downloading from Oracle requires creating an Oracle account, so the archiver couldn't auto-download it even if it were allowed. Sources we can't download are blacklisted by the archiver, and don't count towards the completion percentage on the status page. The current blacklist is: academic/novocraft academic/wehi-weasel development/amd-app-sdk development/decklink-sdk development/jdk development/J-Link development/sqlcl development/sqldeveloper office/treesheets system/displaylink system/oracle-instantclient-devel system/oracle-xe system/oracle-instantclient-basic If you find a file in the archive that shouldn't be there due to its license not allowing redistribution, PLEASE let us know so we can remove and blacklist it. It is not our intention to violate anyone's license. Q: Why does the status page say the by-name tree is missing 4 files, but the by-md5sum tree is 100% complete? A: This is due to a design flaw in the archive structure. We assumed that download filenames would either be unique within an .info file, or else that 2 files with the same filename were in fact the same file. For 4 of the SlackBuilds, this turns out to be a bad assumption. Example: development/p4's .info file has this: DOWNLOAD="https://www.perforce.com/downloads/perforce/r18.1/bin.linux26x86/p4" DOWNLOAD_x86_64="https://www.perforce.com/downloads/perforce/r18.1/bin.linux26x86_64/p4" Notice that both URLs end in "/p4". The directory parts of the URL are different, but the filenames are the same. The archive script successfully downloads these files and stores them in the by-md5 tree in the correct directories. But when it tries to store them in the by-name tree, it's trying to save two files in the same directory with the same name. The second one overwrites the first. The current list of builds affected by this is: academic/ucsc-blat development/p4 development/p4d libraries/p4api Eventually this will get more-or-less fixed: for these builds (and only these), there will probably be separate x86/ and x86_64/ subdirectories (e.g. development/p4/x86/p4). Q: I'm a SlackBuild maintainer, and the download URL for one of my builds has disappeared. Can I use the archive URL as the DOWNLOAD in my .info file? A: Yes, but only as a temporary measure or a last resort. It's better to do one of these: - Find another copy of the source. Try a google search for the exact filename (in "quotes"), or the md5sum. - Host the source yourself, if you have access to a web or ftp server. - Ask on the slackbuilds-users mailing list. Someone will probably volunteer to host the source for you, provided you have a copy of it to send them (and if you don't, hey, there's this handy source archive you can probably get it from...) Using the archive as the DOWNLOAD results in less redundancy. Nobody is currently mirroring the archive that we know of. Ideally, we want every source file to have two working URLs: the original plus the sbosrcarch one. Q: How do I create my own archive? A: Two choices: - Mirror the directory the usual way, using wget or rsync. Using rsync is better! - Get a copy of the sbosrcarch script and run it on your web server. This will be more work on your part, but your archive will be independent: it'll keep updating itself even if the original archive at slackware.uk goes away someday. The script lives here: http://urchlay.naptime.net/repos/sbostuff/plain/sbosrcarch ...or, it'd be better to use git: git clone git://urchlay.naptime.net/sbostuff.git It's written in perl, and has extensive documentation. Run it as "sbosrcarch --help" to see the docs. If you're thinking about running a sbosrcarch instance, please contact me (yalhcru@gmail.com). I've got a list (with only one entry in it) and I'd like it to include all the archives eventually. Also I'm pretty good at troubleshooting, if you're having problems with the script.