blast ftp

NCBI BLAST FTP Site
Tao Tao, Ph.D. User Service NCBI, NLM, NIH
TOC
1. Introduction content 2. File list and content content 2.1 ftp.ncbi.nlm.nih.gov/blast/ general directory content content 2.2 /blast/db/ directory content 2.2.1 /blast/db/FASTA/ subdirectory content 2.3 /blast/demo/ directory content 2.4 /blast/documents/ directory content 2.5 /blast/executables/ directory content 2.5.1 /blast/executables/LATEST/ subdirectory content 2.5.2 /blast/executables/release/ subdirectory content 2.5.3 /blast/executables/snapshot/ subdirectory content 2.6 /blast/matrices/ directory content 2.7 /blast/temp/ directory content 3. Techinical Support
1. Introduction
NCBI BLAST ftp site (ftp.ncbi.nlm.nih.gov) provides software packages for standalone blast, client server blast, and wwwblast packages for various platforms. It also provides commonly used blast databases in preformatted as well as FASTA format. Documents on the blast executables and other related subjects are also available from this site. This file describes the the subdirectories and files found on this ftp site. It also prvoides the basic information on file content and on how the files should be used.
2. File list and content
This section list and describes the files found on the BLAST FTP site. File content for each directory/subdirectory is described in a separate table.
2.1 ftp.ncbi.nlm.nih.gov/blast/ general directory content
The blast ftp directory contains several subdirectories each for a specific set of files.
Table 2.1 ftp.ncbi.nlm.nih.gov/blast/ general directory content
File/Dir Name | Content
blastftp.html | README on FTP site (this file)
db | databases in preformatted or FASTA form
demo | demonstration programs and documents from blast developers
documents | documents for programs in standalone blast, netblast, and wwwblast programs
executables | archives for binary distribution of blast programs
matrices | protein and nucleotide score matrices, only a subset are supported by blast
temp | temporary directory for miscellaneous files
2.2 /blast/db/ directory content
Databases larger than two gigabytes (2 GB) are formatted in multiple volumes, which are named using the "database.##.tar.gz" convention. All relevant volumes are required. An alias file is provided so that the database can be called using the alias name without the extension (.nal or .pal). For example, to call est database, simply use "–d est" option in the commandline (without the quotes). Certain databases are subsets of a larger parental database. For those databases, mask files, rather than actual databases, are provided. The mask file needs the parent database to function properly. The parent databases should be generated on the same day as the mask file. For example, to use swissprot preformatted database, swissprot.tar.gz, one will need to get the nr.tar.gz with the same date stamp. To use the preformatted blast database file, first inflate the file using gzip (unix, linux), WinZip (window), or StuffIt Expander (Mac), then extract the component files out from the resulting tar file using tar (unix, linux), WinZip (Window), or StuffIt Expander (Mac). The resulting files are ready for BLAST.
Table 2.2 /blast/db/ directory content
File/Dir Name | Content
FASTA | subdirectory with databases in FASTA format
blastdb.html | content list of the blast databases
est.*.tar.gz | volumes of the est database, all are needed to reconstitute complete est database
est_human.tar.gz | human est database, a mask file requires all volumes of est to work
est_mouse.tar.gz | mouse est database, a mask file requires all volumes of est to work
est_others.tar.gz | non-human, non-mouse est database, a mask file reqires both volumes of est
gss.*.tar.gz | genomic survery sequence database, all are needed to reconstitute complete gss database
htgs.*.tar.gz | volumes of the htgs database, all are needed to reconstitute complete htgs database
human_genomic.tar.gz | human chromosome database containing concatenated contigs with adjusted gaps represented by N's
nr.*.tar.gz | volumes of non-redundant protein database, all are needed to reconstitute complete nr
nt.*.tar.gz | volumes of the nucleotide "nr" database, all are needed to reconstitute complete nt database
other_genomic.*.tar.gz | volumes of chromosome database for organisms other than human, all are needed to reconstitute complete other_genomic database
pataa.tar.gz | patent protein database
patnt.tar.gz | patent nucleotide database
pdbaa.tar.gz | protein sequence database from pdb entries, a mask file requires all nr.*.tar.gz to function
pdbnt.tar.gz | nucleotide sequence database from pdb entries. They are not the coding sequences for the corresponding protein structure entries!
sts.tar.gz | sequence tag site database
swissprot.tar.gz | swissprot sequence database, last major release. It requires all nr.*.tar.gz to work properly
taxdb.tar.gz | taxonomy id database for use with the above file to retrieve taxonomic information for specific entries
wgs.*.tar.gz | volumes of wgs assembly database, all volumes are needed to reconsititute wgs database
2.2.1 /blast/db/FASTA/ subdirectory content
The FASTA database files are now stored in this subdirectory, it does contain some additional databases that are not available via the NCBI BLAST pages. Due to file size issues, the full est database is not provided. One needs to get the three subsets and concatenate them together to get the complete est database. These databases will need to be formatted using formatdb program found in the standalone blast executable package. The recommended commandlines to use are: formatdb –i input_db –p F –o T for nucleotide formatdb –i input_db –p T –o T for protein For additional information on formatdb, please see the formatdb.html at: /blast/documents/formatdb.html
2.2.1 blast/db/FASTA/ subdirectory content
File/Dir Name | Content
alu.a.gz | proteins translated from alu.n
alu.n.gz | alu repeat sequences
drosoph.aa.gz | Drosophila protein from genome annotation
drosoph.nt.gz | Drosophila genome
ecoli.aa.gz | E.coli K-12 proteins from genome annotation
ecoli.nt.gz | E.coli K-12 genomic contigs
est_human.gz | human subset of the est database
est_mouse.gz | mouse subset of the est database
est_others.gz | subset of est other than human or mouse entries
gss.gz | Genomic Survey Sequences (mostly BAC ends)
htgs.gz | High Throughput Genomic Sequences
human_genomic.gz | Human chromosomes formed by concatenating genomic contig assemblies (NT_######) and adjusting the gaps with N’s
igSeqNt.gz | Immunoglobulin nucleotide sequences
igSeqProt.gz | Immunoglobulin protein sequences
mito.aa.gz | protein from the annotated mitochondrial genomes
mito.nt.gz | mitochondrial genomes
month.aa.gz | protein sequences released or updated in the past 30 days
month.est_human.gz | human subset of EST released/updated in the past 30 days
month.est_mouse.gz | mosue subset of EST released/updated in the past 30 days
month.est_others.gz | non-human non-mouse EST, released or updated in the past 30 days
month.gss.gz | gss entries released/updated in the past 30 days
month.htgs.gz | htgs entries released/updated in the past 30 days
month.nt.gz | subset of nt released/updated in the past 30 days
nr.gz | non-redundant protein sequence database
nt.gz | nucleotide database from GenBank excluding the batch division htgs, est, gss,sts, pat divisions, and wgs entries. Not non-redundant
other_genomic.gz | Chromosome entries other than human
pataa.gz | Patent protein sequence database
patnt.gz | Patent nucleotide sequence database
pdbaa.gz | protein sequences for pdb entries
pdbnt.gz | nucleotide entries for pdb entries. They are NOT the coding sequence forthe corresponding protein entries
sts.gz | Sequence Tag Sites database
swissprot.gz | swissprot database, last major release
vector.gz | vector sequences from synthetic (syn) division of GenBank
wgs.gz | Whole Genome Shotgun sequence assembly
yeast.aa.gz | protein translations from yeast genome annotation
yeast.nt.gz | yeast genomic sequence
2.3 File content for /blast/demo/ directory
This directory contains some technical presentations from the BLAST developers along with some demo tools or documentation relevant to BLAST.
Table 2.3 File content for /blast/demo/ directory
File/Dir Name | Content
README.blast_demo | readme for blast_demo package
README.first | readme for this directory
README.parse_blast_xml | readme for parse_blast_xml package
benchmark | Package with sample database and query for gauging the performance of BLAST on different platforms
blast_demo.tar.gz | blast_demo package on blast db, blast object, and reformating blast alignment from blastobj file
blast_programming.ppt | PowerPoint presentation on BLAST programing
ieee_blast.final.ppt | PowerPoint presentation (IEEE conference)
ieee_talk.pdf | PDF file for presentation (IEEE conference)
ieee_talk.pdf | Above IEEE presentation in PDF format
old | Containing an older version of blast_demo.tar.gz
parse_blast_xml.tar.gz | demo package on parsing xml styled blast output
splitd.ppt | PowerPoint presentation on NCBI BLAST server’s splitd implementation
test_suite.tar.gz | test package
2.4 File content for /blast/documents/ directory
This directory contains copies of the documentation on different BLAST programs distributed from this ftp site under the /blast/executables/ directory. blast.txt also contains detailed release history.
2.4 File content for /blast/documents/ directory
File/Dir Name | Content
bl2seq.html | readme for bl2seq (standalone "Align Two Sequences")
blast-sc2004.pdf | POSTER describing NCBI splitd system
blast.html | General readme for blast setup
blastall.html | readme for blastall and blastpgp
blastclust.html | readme for blastclust
blastdb.html | readme for blast databases
blastftp.html | readme for blast ftp site (this document)
blastpgp.html | readme for blastpgp, standalone PSI-BLAST
developer | subdirectory with additional documentation: blast_seqalign.txt: describing seqalign function readdb.txt: describing readdb function scoring.pdf: describing scoring systems used by BLAST urlapi.txt: a short introduction on BLAST URL API
fastacmd.html | readme for fastacmd, a sequence retrieval tool
filter.html | readme on filter strings and their functions
formatdb.html | readme for formatdb, which converts fasta sequences into blast database
formatrpsdb.html | readme for formatrpsdb, which converts scoremat into rpsblast database
history.html | history of changes/bug fixes introduced to blast package
impala.html | readme for impala
index.html | list of this group of files/subdirs
megablast.html | readme for megablast
netblast.html | readme for netblast (blastcl3)
rpsblast.html | readme for rpsblast
seedtop.html | readme for seedtop, a standalone pattern match tool
web_blast.pl | Example perl script for doing URLAPI blast
xml | subdirectory with .dtd and .mod field description files for blast xml output NCBI_BlastOutput.dtd: dtd file for blast xml output NCBI_BlastOutput.mod: mod file for blast xml output NCBI_Entity.mod: mod file for NCBI xml file README.blxml: readme on blast xml output
2.5 File content for /blast/executables/ directory
This directory contains several subdirectories each for a specific subsets of executable BLAST programs.
2.5.1 File content for /blast/executables/LATEST/ subdirectory
This directory contains the latest official release of precompiled BLAST executable programs. The binaries can be divided into three groups. blast initialed files are equivalent. They contain the standalone command line blast binary programs for different platforms. Users need this package to set up BLAST locally. It also provides the tools necessary to prepare custom databases and retrieve sequences from these prepared databases. The netblast initialed archives the blastcl3 program which functions by formulating BLAST search locally first before forwarding the search to NCBI blast server for process. The search results returned by NCBI BLAST server is saved to an user-specified file on local computer disk. Users do not need to maintain local databases, nor can they search custom databases locally. The wwwblast initialed archive contains the web pages with embedded blast search forms similar to that of NCBI. They can process the BLAST request submitted through web and search against local set of databases and return the result to a browser window. wwwblast is now in sync with the NCBI toolkit and the two above two packages. Installation requires existing web server (apache) setup.
Table 2.5.1 File content for /blast/db/executables/LATEST/ directory
File/Dir Name | Content
blast-x.y.z-axp64-tru64.tar.gz | command line blast for COMPAQ/HP alpha machine (OSF 5.1 and above)
blast-x.y.z-ia32-freebsd.tar.gz | command line blast for PC running freeBSD
blast-x.y.z-ia32-linux.tar.gz | command line blast for PC running Linux
blast-x.y.z-ia32-solaris.tar.gz | command line blast for PC runnig solaris
blast-x.y.z-ia32-win32.exe | command line blast for PC running windows
blast-x.y.z-mips64-irix.tar.gz | command line blast for 64-bits SGI machine
blast-x.y.z-ppc32-macosx.tar.gz | command line blast for MacOSX
blast-x.y.z-sparc64-solaris.tar.gz | command line blast for Sparc running Solaris
blast-x.y.z-x64-linux.tar.gz | command line blast for 64-bits Linux
ncbi.tar.gz | NCBI toolkit
ncbiz.exe | NCBI toolkit for PC
netblast-x.y.z-axp64-tru64.tar.gz | netblast for COMPAQ/HP alpha machine (OSF 5.1 and above)
netblast-x.y.z-ia32-freebsd.tar.gz | netblast for PC running freeBSD
netblast-x.y.z-ia32-linux.tar.gz | netblast for PC running Linux
netblast-x.y.z-ia32-solaris.tar.gz | netblast for PC runnig solaris
netblast-x.y.z-ia32-win32.exe | netblast for PC running windows
netblast-x.y.z-mips64-irix.tar.gz | netblast for 64-bits SGI machine
netblast-x.y.z-ppc32-macosx.tar.gz | netblast for MacOSX
netblast-x.y.z-sparc64-solaris.tar.gz | netblast for Sparc running Solaris
netblast-x.y.z-x64-linux.tar.gz | netblast for 64-bits Linux
wwwblast-x.y.z-axp64-tru64.tar.gz | web server blast for COMPAQ/HP alpha machine (OSF 5.1 and above)
wwwblast-x.y.z-ia32-freebsd.tar.gz | web server blast for PC running freeBSD
wwwblast-x.y.z-ia32-linux.tar.gz | web server blast for PC running Linux
wwwblast-x.y.z-ia32-solaris.tar.gz | web server blast for PC runnig solaris
wwwblast-x.y.z-mips64-irix.tar.gz | web server blast for 64-bits SGI machine
wwwblast-x.y.z-ppc32-macosx.tar.gz | web server blast for MacOSX
wwwblast-x.y.z-sparc64-solaris.tar.gz | web server blast for Sparc running Solaris
wwwblast-x.y.z-x64-linux.tar.gz | web server blast for 64-bits Linux
NOTE: Currently, we do not have access to IBM AIX server to make the binary for that platform. Also, we do not make wwwblast package of window.
2.5.2 /blast/executables/release/ subdirectory content
This directory contains past major releases of BLAST, as far back as version 2.0.10. Each release is in its own subdirectory.
2.5.2 File content for /blast/executables/release/ subdirectory
File/Dir Name | Content
2.0.10 | Release version 2.0.10
2.0.11 | Release version 2.0.11
2.0.12 | Release version 2.0.12
2.0.13 | Release version 2.0.13
2.0.14 | Release version 2.0.14
2.0.7 | Release version 2.0.7
2.0.8 | Release version 2.0.8
2.0.9 | Release version 2.0.9
2.1.2 | Release version 2.1.2
2.1.3 | Release version 2.1.3
2.2.10 | Release version 2.2.10
2.2.11 | Release version 2.2.11
2.2.12 | Release version 2.2.12
2.2.13 | Release version 2.2.13
2.2.3 | Release version 2.2.3
2.2.4 | Release version 2.2.4
2.2.5 | Release version 2.2.5
2.2.6 | Release version 2.2.6
2.2.7 | Release version 2.2.7
2.2.8 | Release version 2.2.8
2.2.9 | Release version 2.2.9
2.5.3 /blast/executables/snapshot/ subdirectory content
This directory contains compilation of blast packages with bug fixes in between major versioned releases. The subdirectories are named with date. There was not snapshot or patch since the end of 2004, which as for release 2.2.10. Available old patches will not be listed here.
2.6 /blast/matrices directory content
This directory contains the scoring matrices, which are files that can be used by BLAST alignment assessment. The file are text files with special format that can be viewed directly by a browser. For valid statistical analysis, blastn uses only identity matrix and blastp only supports a limited subset of the BLOSUM and PAM matrices: BLOSUM45, BLOSUM62, BLOSUM80, plus PAM30 and PAM70.
2.7 /blast/temp/ directory content
For temporary file storage and miscellaneous files or tools. Current empty.
3. Techinical Support
Additional questions/comments on this ftp site should be directed to NCBI blast-help group at: blast-help@ncbi.nlm.nih.gov Other questions on general NCBI resources should be directed to: info@ncbi.nlm.nih.gov