README file for KLibido - the KDE Linux Binaries Downloader 

Version 0.2.5 - 27/Feb/2005


* What it is:
KLibido is a KDE program to download encoded articles from the usenet news
service, using the nntp protocol. It supports multiple servers, multiple
download threads per server, automatic joining and decoding of articles.

* What it's not:
KLibido is not a NewsReader. It doesn't let you easily display the articles -
only their subject, and it discards all non-binary posts. If you want a nice
newsreader for KDE, try KNode.

* What works:
 - Getting the group list and subscribe to newsgroups
 - Getting headers, sorting, filtering by subject, state and "completeness" 
 - downloading and decoding of binary posts (including yEnc).
 - Queue management: it's possible to drag and drop items around in the download
   queue to change the order of download, pause the queue, pause and cancel the
   items in the queue.
 - Server priorities and balancing
 - Requeueing: if an article fails on a server, it is automatically tried on
   the other servers.
 - Viewing nfo's, covers, et al inside the program using kparts.
 - Group categories and aliases
 - .nzb support

* What doesn't work:
 - No documentation :)
 - There are probably a lot of bugs. Having said that, it quite works, and I've
   used it to download tens of gigabytes.

 
* How does it do it:
 - KLibido uses kde and the kde libraries for the interface, Berkeley DB for
   storing the newsgroups' headers, and UUDeview for decoding the posts.
 
Requirements:
 - Kde/Qt 3.2 or greater
 - Berkeley Db 4.x (http://www.sleepycat.com/products/db.shtml) with C++
   support. If you compile it from source, remember to  pass the "--enable-cxx"
   option to configure.
 - UUdeview library version 0.5.20 (http://www.fpx.de/fp/Software/UUDeview/). 
   If you're a Gentoo user, emerge dev-libs/uulib.

* How to compile:

- You can grab the source from sourceforge (go to
http://www.sf.net/projects/klibido). It compiles as usual:


./configure --prefix=<your kde prefix>
make
make install

You can find out your kde prefix running

kde-config --prefix

PLEASE NOTE: Lots of problems/crashes have been reported when using 
optimizations. Please be conservative.

If you want to grab the source from CVS, go to
http://sourceforge.net/cvs/?group_id=114928 and follow the instructions. 

PLEASE NOTE: I try to commit only working code, but the CVS can be very
unstable at times. Always read the Changelog before compiling/using.

* Binary Packages:

On the program's homepage there are links/instructions for downloading binary packages for some distributions (Mandrake, Fedora, SuSE, Debian, etc...). See http://klibido.sf.net/ if you don't want to compile the source yourself.

* Getting started:

 Type "klibido" to start the program. Sorry, no fancy wizards at the moment. To
get it working:
  - Configure the directories (a default is provided)
  - Add your servers (Server->new server)
  - Get the group list (Server->get list of groups)
  - Subscribe to some groups (newsgroup->subscribe to newsgroup)
  - NewsGroup->update subscribed groups
  - Enjoy :)
  
* A note about error model: 

KLibido divides errors in two branches, fatal (i.e.: article not found) and not
fatal (i.e.: timeout). Fatal errors cause the download of the item to fail, 
while non-fatal errors trigger a retry.

In the server options, there are three values that control the connection 
timeout/errors: 
1. Server timeout: this is the value of timeout for the server (in seconds), 
i.e.: if no response/data come from the server in "timeout" seconds, a timeout 
error is triggered (and the item is generally retried)
2. Retries: this value only applies to post downloads (i.e.: not to headers 
updates) and represents the number of times a download is retried in the event 
of a *fatal* error. Example: KLibido tries to download an article. The server 
answers "article not found". The download is retried "Retries" times 
(default: 3). Why, you may ask? Because some servers are lame/overloaded/whatever,
and when you try to download an article you get "article not found" the first time, 
but if you retry in a few seconds, the download succeeds. If your servers don't 
show this behavior, put "retries" to 0.
3. Thread timeout: this is the timeout (in minutes) for the connection thread. 
When a thread fails to successfully connect/login to a server for "thread_timeout" 
minutes, the thread is paused and must be restarted manually (via the context menu). 
This is considered a communication error, so it doesn't cause any item in the queue 
to fail. 

More in depth: when a thread gets a login error, it is paused for n seconds. 
"n" depends on the number of *consecutive* fails: first time 1 seconds, second 
5 seconds, third 10 seconds, then 30, then one minute. After that, the thread 
is always paused for one minute. After "thread_timeout" minutes, the thread is 
put in paused state, and doesn't try to connect anymore until the user restarts 
it manually. Why? Because a login error can have to much causes:
- network is down
- wrong login/password
- thread limit exceeded
- download limit exceeded
- etc...
Servers often return custom error codes, so in the event of a failed login there 
is no easy/automated way to know if the login failed because of a wrong password, 
exceeded daily download limit, too many connected threads, or whatever. So the best 
thing to do (IMHO) is to pause the thread and retry later.

* The story behind it:
 
The main goals I had in mind while writing it were:
 - Keeping memory usage as low as possible. Update/load/browse large groups
   (groups with over one million of articles) without using 1 Gb of RAM and
   swapping for days.
 - Multiple server support
 - Multiple download threads support
 - Fine-grained manipulation of the download queue.

I hope to have succeeded.

* Who am I:
 - Does anyone care? :-]

* Credits
 - KNode and Pan developers for letting me steal their icons :)
 - Sleepycat for the great Berkeley Db
 - Frank Pilhofer for the UUDeview Library.  
 - The KDevelop team for the great IDE they've made.
 - The KDE developers for the great GUI. 
 - David Pye (.nzb parsing, yy/uu decoder)


* Related software you will find on the 'net:
 - nget - http://nget.sourceforge.net/ - The *best* command-line nntp grabber.
 - BNR - http://www.bnr2.org/ - Multi-platform nntp grabber, available for Linux
   and Windows.
 - Pan - http://pan.rebelbase.com/ - Great newsreader with binary skills :).
 - Knzb - http://dmp.org.uk/linux/knzb/ - KDE .nzb (Nezbin) downloader
   
 (C) 2004/2005 Alessandro "Bauno" Bonometti, <bauno_at_inwind.it>
