diff options
| -rw-r--r-- | sbosrcarch.faq | 329 |
1 files changed, 329 insertions, 0 deletions
diff --git a/sbosrcarch.faq b/sbosrcarch.faq new file mode 100644 index 0000000..19b30ac --- /dev/null +++ b/sbosrcarch.faq @@ -0,0 +1,329 @@ +Q: What is sbosrcarch? + +A: sbosrcarch is "The SlackBuilds.Org Source Archive". It contains copies + of the source files listed in the .info files for all (or almost all) + the builds on SlackBuilds.org. + + sbosrcarch is also the name of the software that created and maintains + the archive (more about this later, near the end of this FAQ). + +Q: What is sbosrcarch for? + +A: It's intended to be a backup location for source files that can't be + downloaded. This happens mainly for these reasons: + + - The upstream web site goes down, is moved, or has connectivity + issues (intermittent or long-term). + - Upstream moves or removes the source, when they release a new version. + + Also, the archive is hosted on a fast, well-connected host. Sometimes + you might choose to use the archive just for faster downloads. + + A side benefit of the archiving process is that the archive maintenance + software produces a log of failed downloads, which can then be sent + to the slackbuilds-users mailing list and/or build maintainer so it + can be fixed quickly. + +Q: Who is responsible for sbosrcarch? + +A: The archive server is operated by Darren Austin, aka "Tadgy" + on Freenode IRC. The archive script was written by B. Watson, aka + "Urchlay" on Freenode. Both of us keep an eye on the logs and keep the + archive healthy. + + The best way to contact us is using an IRC client to connect to + Freenode and join the ##slackware or ##slackbuilds channel. + + We can also be reached by email: + + B. Watson <yalhcru@gmail.com> + TODO: make sure Tadgy is OK with his email being here! + + Please read this entire FAQ before asking us questions. Chances are, + you'll find the answer here. If not, or if the answer isn't clear + enough, we'll be happy to help. + + Note that the SlackBuilds.org team is NOT responsible for the + archive. PLEASE don't bother them with questions about sbosrcarch, + they're already busy enough maintaining the actual SlackBuilds site! + Same goes for individual build maintainers. + +Q: Why create a giant archive like this? Isn't it better to fix the + SlackBuilds whose sources can't be downloaded? + +A: Sort-of. Yes, if a SlackBuild references a no-longer-existing + source download URL, it should be updated. Usually the SlackBuild + maintainer is responsible for this. Sometimes the SBo admins take + care of it instead. Sometimes, it takes longer than expected to + update a SlackBuild: the new version uses a different build system, + or requires some dependency to be updated first, or the maintainer + is too busy with Real Life and can't spare the time just at the moment. + + Once the build is updated, it still doesn't appear instantly on the + site. It has to sit in the "pending" queue until it's been reviewed by + the admins, and then in the "ready" queue until the next public update. + + The SBo update process is complex, and requires coordination between + the various admins. Generally this means that site updates ("Public + www update" in the git log) only happen once a week. + + During the time it takes for the SlackBuild to get updated for the + new download URL (and possibly new version), users won't be able to + download the source as listed on the SBo site. + + That's what the archive is mainly intended for. It's a fallback, + a stop-gap solution, that allows builds to keep working during the + period between the source disappearing and the build being updated. + Usually this is only a week or less, but sometimes things slip through + the cracks... + +Q: How do I use the archive? + +A: Several answers here: + + - Using a tool that supports the archive, such as sbopkg or sbotools. + + This is by far the easiest way: they automatically use the archive + if they need to, without you having to do any extra work. + + - Manually with a web browser. The easy way is to start at: + + http://slackware.uk/sbosrcarch/by-name/ + + ...which shows a list of category directories (academic, accessibility, + audio, etc). Choose a category, then within the category + you'll see a list of build name directories. Each of these will + contain the source file(s) for the build. + + Example: you can't download the source to system/atari800, so + you go to the by-name page, click on "system", then "atari800". + There you'll see the file you wanted, atari800-3.1.0.tar.gz (unless + it's been updated since I wrote this). + + - With a download tool like wget or curl. You could do this using the + same by-name tree as you would for manual lookups, but it's better to + do this by md5sum. The base URL for this is: + + http://slackware.uk/sbosrcarch/by-md5/ + + In the build's .info file, take the 'filename' part of each download + URL. Example: "atari800-3.1.0.tar.gz", where the link is + http://downloads.sourceforge.net/project/atari800/atari800/3.1.0/atari800-3.1.0.tar.gz + + Now take the MD5SUM (or MD5SUM_x86_64 if you're using DOWNLOAD_x86_64), + and use the first two characters as subdirectory names, followed by the + full md5sum. Example: we have + + MD5SUM="354f8756a7f33cf5b7a56377d1759e41" + + in the .info file. The directory for this would be: + + 3/5/354f8756a7f33cf5b7a56377d1759e41 + + Add this to the base URL and get: + + http://slackware.uk/sbosrcarch/by-md5/3/5/354f8756a7f33cf5b7a56377d1759e41/ + + Now add the filename part from DOWNLOAD or DOWNLOAD_x86_64, and you get: + + http://slackware.uk/sbosrcarch/by-md5/3/5/354f8756a7f33cf5b7a56377d1759e41/atari800-3.1.0.tar.g + + This is the exact URL for the file, if it's actually present in the + archive. Most likely, it will be, and your download will succeed. If + the download fails, the file's not in the archive. + + Of course, all these steps should be automated. You'll end up writing + a script in your favorite language to do the job. Or: + + - Using the sbosrc script + + Same as above, except someone's already written it for you. Download + it here: + + http://urchlay.naptime.net/repos/sbostuff/plain/sbosrc + + ...or, it'd be better to use git: + + git clone git://urchlay.naptime.net/sbostuff.git + + Make it executable (chmod +x) and place it somewhere on your $PATH, + such as /usr/local/bin. + + Whenever you need to download something from the archive, change + to the directory containing the .info file (same place as the + .SlackBuild) and just run: + + sbosrc + + ...which will check the current architecture (32-bit or 64-bit), + parse the info file, calculate the URL as above, and download the + file to the current directory. + +Q: I need a specific older version of a source file, not the latest + version that's packaged on SBo. Will the archive have it? + +A: Probably not. Old versions don't disappear immediately when new + ones are archived, but they do get purged monthly... or, almost: + old files are deleted on the 30th of every month, and February is + only 28 or 29 days long! + + Use the by-md5 tree if you're looking for an old version, since some + builds use unversioned filenames (new one will overwrite the old, + in the by-name tree). + + If you know the exact filename and/or md5sum, you can always try a + google search for them. Use "quotes" around the filename. + +Q: How do I know it's safe to use files downloaded from the archive? + +A: The same way you know it's safe to use any file you downloaded for + use with a SlackBuild: check the downloaded file's md5sum against + the MD5SUM line in the build's .info file. + +Q: How do I use the archive with automated tools such as sbopkg and sbotools? + +A: For sbopkg and sbotools, you just run them normally. They'll automatically + search the archive, if a source download fails. + +Q: How complete is the archive? + +A: Currently (2018-06-26), the by-md5 tree is 100% complete. This does + NOT count blacklisted sources (see next question). + + For a more up-to-date answer, see the archive status page: + + http://slackware.uk/sbosrcarch/STATUS + + This gets updated nightly. + +Q: Why are some sources missing from the archive? + +A: Multiple answers: + + - The archiver couldn't download the file. Maybe the site was down + when it tried, or the upstream developers removed the file. Generally + this will require the build's maintainer to fix the .info file or + update the SlackBuild to a newer version (that actually exists). + In some cases, the archive operator will find the file and manually + add it to the archive. + + - The archiver downloaded the file, but the download's md5sum doesn't + match. The build maintainer will have to fix the .info file. We + won't archive any files we can't verify by md5sum. + + - There is some software that can't be automatically downloaded + (requires account creation on the upstream site) or whose license + doesn't allow us to redistribute it. + + The classic example of both is development/jdk: Oracle's license + requires that users download the file directly from their site and + doesn't allow us (or anyone else) to offer it for download. Also, + downloading from Oracle requires creating an Oracle account, so + the archiver couldn't auto-download it even if it were allowed. + + Sources we can't download are blacklisted by the archiver, and + don't count towards the completion percentage on the status page. + The current blacklist is: + + academic/novocraft + academic/wehi-weasel + development/amd-app-sdk + development/decklink-sdk + development/jdk + development/J-Link + development/sqlcl + development/sqldeveloper + office/treesheets + system/displaylink + system/oracle-instantclient-devel + system/oracle-xe + system/oracle-instantclient-basic + + If you find a file in the archive that shouldn't be there due to + its license not allowing redistribution, PLEASE let us know so we + can remove and blacklist it. It is not our intention to violate + anyone's license. + +Q: Why does the status page say the by-name tree is missing 4 files, but + the by-md5sum tree is 100% complete? + +A: This is due to a design flaw in the archive structure. We assumed that + download filenames would either be unique within an .info file, or else + that 2 files with the same filename were in fact the same file. + + For 4 of the SlackBuilds, this turns out to be a bad assumption. Example: + development/p4's .info file has this: + + DOWNLOAD="https://www.perforce.com/downloads/perforce/r18.1/bin.linux26x86/p4" + DOWNLOAD_x86_64="https://www.perforce.com/downloads/perforce/r18.1/bin.linux26x86_64/p4" + + Notice that both URLs end in "/p4". The directory parts of the URL are + different, but the filenames are the same. + + The archive script successfully downloads these files and stores them + in the by-md5 tree in the correct directories. But when it tries to + store them in the by-name tree, it's trying to save two files in the + same directory with the same name. The second one overwrites the first. + + The current list of builds affected by this is: + + academic/ucsc-blat + development/p4 + development/p4d + libraries/p4api + + Eventually this will get more-or-less fixed: for these builds (and only + these), there will probably be separate x86/ and x86_64/ subdirectories + (e.g. development/p4/x86/p4). + +Q: I'm a SlackBuild maintainer, and the download URL for one of my builds + has disappeared. Can I use the archive URL as the DOWNLOAD in my .info + file? + +A: Yes, but only as a temporary measure or a last resort. + + It's better to do one of these: + + - Find another copy of the source. Try a google search for the exact + filename (in "quotes"), or the md5sum. + + - Host the source yourself, if you have access to a web or ftp server. + + - Ask on the slackbuilds-users mailing list. Someone will probably + volunteer to host the source for you, provided you have a copy of + it to send them (and if you don't, hey, there's this handy source + archive you can probably get it from...) + + Using the archive as the DOWNLOAD results in less redundancy. Nobody + is currently mirroring the archive that we know of. Ideally, we want + every source file to have two working URLs: the original plus the + sbosrcarch one. + +Q: How do I create my own archive? + +A: Two choices: + + - Mirror the directory the usual way, using wget or rsync. Using + rsync is better! + + - Get a copy of the sbosrcarch script and run it on your web server. + This will be more work on your part, but your archive will be + independent: it'll keep updating itself even if the original archive + at slackware.uk goes away someday. + + The script lives here: + + http://urchlay.naptime.net/repos/sbostuff/plain/sbosrcarch + + ...or, it'd be better to use git: + + git clone git://urchlay.naptime.net/sbostuff.git + + It's written in perl, and has extensive documentation. Run it as + "sbosrcarch --help" to see the docs. + + If you're thinking about running a sbosrcarch instance, please + contact me (yalhcru@gmail.com). I've got a list (with only one + entry in it) and I'd like it to include all the archives eventually. + Also I'm pretty good at troubleshooting, if you're having problems + with the script. |