trunk/doc/Search.pod

=head1 BackupPC search and archival extension

BackupPC search and archival extension is used to:

=over 4

=item *

find files in C<pool> by substring of filename

=item *

see state of one or all of your shares in some particular point in time
(file creation date on clients or backup date is supported as filter)

=item *

archive selected backups (per host, share and increment) to archival
medium (CD, DVD, tape etc.) with rich descriptions, archival locations,
multiple copies and so on.

=back

=head2 Requirements

This extension is based on PostgreSQL RDBMS and Hyper Estraier full-text
search engine (and it's perl bindings).

If you are using Debian, you are in luck. All required packages are
allready part of C<unstable> distribution and can be installed with:

  # apt-get install postgresql
  # apt-get install hyperestraier libestraier-dev libqdbm-dev

Now you can skip to installation of L<Hyper Estraier perl bindings> below.

If you don't have pre-packaged binaries for your installation, you will
need to install additional packages by hand.

=head3 PostgreSQL

You can use packages provided by your distribution or follow installation
instructions on PostgreSQL site.

=head3 QDBM

First, you need qdbm on which Hyper Estraier depends. Installation is simple.

  $ tar xvfz qdbm-1.8.31.tar.gz 
  $ cd qdbm-1.8.31
  $ ./configure --enable-zlib
  $ make
  $ sudo make install

=head3 Hyper Estraier

Also quite simple.

  $ tar xvfz hyperestraier-0.5.4.tar.gz
  $ cd hyperestraier-0.5.4
  $ ./configure
  $ make
  $ sudo make install

Then you will have to install perl bindings for Hyper Estraier.

=head3 Hyper Estraier perl bindings

Just use C<cpan> shell to install C<Search::Estraier> module

  $ sudo cpan Search::Estraier

There was awaful long procedure about installing perl C bindings, but it's all gone now.

=head3 CPAN modules

You will also need a few additional cpan modules

=over 4

=item File::Pid
=item File::Which
=item File::Path
=item File::Slurp
=item Search::Estraier

=item Spreadsheet::WriteExcel

=item Term::Menus
=item XML::Writer
=item Algorithm::Diff
=item Archive::Tar::Stream

=back

Last module, C<Spreadsheet::WriteExcel> is needed only if you want to use
C<BackupPC_xls_report> to generate Excel reports from your backup data.

Easiest way to install them is using C<cpan> shell.

  $ sudo cpan File::Pid Spreadsheet::WriteExcel ...

=head1 Creation of initial database

Once you have all components installed, you should initially create data
about increments in RDBMS and full-text search engine index.

First, edit C<config.pl> and setup C<SearchDSN> to valid perl DSN (for example,
C<dbi:Pg:dbname=backuppc> and C<SearchUser> to database user which has priviledges
over that database. You might need to add additional directives in PostgreSQL's
C<pg_hba.conf> so that selected user can be connected without password (I know, it's not
perfect, but I trust local users on backuppc machine).

Then, create new database for backuppc.

  $ createdb backuppc

Then invoke C<BackupPC_updatedb> for the first time with argument to create database
schema:

  $ sudo -u backuppc /data/backuppc/bin/BackupPC_updatedb -c

You can also force full-text reindex by C<-i> flag. This will add all files which are in
database but for some reason are missing in full-text index.

Now, you can setup cron job which will daily check your backups and update database
and full-text index. This is as simple as invoking

  /data/backuppc/bin/BackupPC_updatedb

As C<backuppc> user and redirecting output to log file.

B<Documentation is still incomplete>.

Now that you finished installation, you can select new options from
menu on the left and example search and archival.

=head1 Reporting in Excel

Often, it is useful to be able to present report about your BackupPC hosts, number
of snapshots, total size and other useful info. While all those information can
be accessed using web browser, for analysis it's useful to have them in spreadsheet
form. With this data, you can monitor changes on your backup cycle, effects of changes
on your server or network to your BackupPC installation and so on.

You can create Excel spreadsheet (which works perfectly with Gnumeric also) using
following command:

  $ sudo -u backuppc /data/backuppc/bin/BackupPC_xls_report

=head1 Additional requirements

You will also need several other command-line utilities to make
C<BackupPC_burnArchiveCLI> run. Those include:

=over 4

=item mkisofs

=item cdrecord

Or equivavalent utility, on Debian you might want to use C<dvdrecord>.

=item eject

Please make a symlink from C</dev/cdrom> to you CD/DVD burner.

=item split

=back

=head1 Troubleshoting

=head2 md5sum problems on Debian

If you are running Debian stable and expiriencing error message like

 -: No such file or directory

your C<md5sum> comes from C<coreutils> (which doesn't know how to handle C<->
as input) as opposed to C<md5sum> which comes with C<textutils> (which does).

Fix is simple, correct md5sum is allready installed, you just have to do:

 sudo cp /usr/bin/md5sum.textutils /usr/bin/md5sum

=head1 Related projects

BackupPC allready has archival host which might suit your needs better (and
it's quite easier to install).

=over 4

=item PostgreSQL

L<http://www.postgresql.org/>

=item HyperEstraier

L<http://hyperestraier.sourceforge.net/>

=back

=head1 Authors

Ivan Klaric C<< <iklaric@gmail.com> >>

Dobrica Pavlinusic C<< <dpavlin@rot13.org> >>

=head1 Licence

This extension is released under GPL licence, same as BackupPC.

1	=head1 BackupPC search and archival extension
2
3	BackupPC search and archival extension is used to:
4
5	=over 4
6
7	=item *
8
9	find files in C<pool> by substring of filename
10
11	=item *
12
13	see state of one or all of your shares in some particular point in time
14	(file creation date on clients or backup date is supported as filter)
15
16	=item *
17
18	archive selected backups (per host, share and increment) to archival
19	medium (CD, DVD, tape etc.) with rich descriptions, archival locations,
20	multiple copies and so on.
21
22	=back
23
24	=head2 Requirements
25
26	This extension is based on PostgreSQL RDBMS and Hyper Estraier full-text
27	search engine (and it's perl bindings).
28
29	If you are using Debian, you are in luck. All required packages are
30	allready part of C<unstable> distribution and can be installed with:
31
32	# apt-get install postgresql
33	# apt-get install hyperestraier libestraier-dev libqdbm-dev
34
35	Now you can skip to installation of L<Hyper Estraier perl bindings> below.
36
37	If you don't have pre-packaged binaries for your installation, you will
38	need to install additional packages by hand.
39
40	=head3 PostgreSQL
41
42	You can use packages provided by your distribution or follow installation
43	instructions on PostgreSQL site.
44
45	=head3 QDBM
46
47	First, you need qdbm on which Hyper Estraier depends. Installation is simple.
48
49	$ tar xvfz qdbm-1.8.31.tar.gz
50	$ cd qdbm-1.8.31
51	$ ./configure --enable-zlib
52	$ make
53	$ sudo make install
54
55	=head3 Hyper Estraier
56
57	Also quite simple.
58
59	$ tar xvfz hyperestraier-0.5.4.tar.gz
60	$ cd hyperestraier-0.5.4
61	$ ./configure
62	$ make
63	$ sudo make install
64
65	Then you will have to install perl bindings for Hyper Estraier.
66
67	=head3 Hyper Estraier perl bindings
68
69	Just use C<cpan> shell to install C<Search::Estraier> module
70
71	$ sudo cpan Search::Estraier
72
73	There was awaful long procedure about installing perl C bindings, but it's all gone now.
74
75	=head3 CPAN modules
76
77	You will also need a few additional cpan modules
78
79	=over 4
80
81	=item File::Pid
82	=item File::Which
83	=item File::Path
84	=item File::Slurp
85	=item Search::Estraier
86
87	=item Spreadsheet::WriteExcel
88
89	=item Term::Menus
90	=item XML::Writer
91	=item Algorithm::Diff
92	=item Archive::Tar::Stream
93
94	=back
95
96	Last module, C<Spreadsheet::WriteExcel> is needed only if you want to use
97	C<BackupPC_xls_report> to generate Excel reports from your backup data.
98
99	Easiest way to install them is using C<cpan> shell.
100
101	$ sudo cpan File::Pid Spreadsheet::WriteExcel ...
102
103	=head1 Creation of initial database
104
105	Once you have all components installed, you should initially create data
106	about increments in RDBMS and full-text search engine index.
107
108	First, edit C<config.pl> and setup C<SearchDSN> to valid perl DSN (for example,
109	C<dbi:Pg:dbname=backuppc> and C<SearchUser> to database user which has priviledges
110	over that database. You might need to add additional directives in PostgreSQL's
111	C<pg_hba.conf> so that selected user can be connected without password (I know, it's not
112	perfect, but I trust local users on backuppc machine).
113
114	Then, create new database for backuppc.
115
116	$ createdb backuppc
117
118	Then invoke C<BackupPC_updatedb> for the first time with argument to create database
119	schema:
120
121	$ sudo -u backuppc /data/backuppc/bin/BackupPC_updatedb -c
122
123	You can also force full-text reindex by C<-i> flag. This will add all files which are in
124	database but for some reason are missing in full-text index.
125
126	Now, you can setup cron job which will daily check your backups and update database
127	and full-text index. This is as simple as invoking
128
129	/data/backuppc/bin/BackupPC_updatedb
130
131	As C<backuppc> user and redirecting output to log file.
132
133	B<Documentation is still incomplete>.
134
135	Now that you finished installation, you can select new options from
136	menu on the left and example search and archival.
137
138	=head1 Reporting in Excel
139
140	Often, it is useful to be able to present report about your BackupPC hosts, number
141	of snapshots, total size and other useful info. While all those information can
142	be accessed using web browser, for analysis it's useful to have them in spreadsheet
143	form. With this data, you can monitor changes on your backup cycle, effects of changes
144	on your server or network to your BackupPC installation and so on.
145
146	You can create Excel spreadsheet (which works perfectly with Gnumeric also) using
147	following command:
148
149	$ sudo -u backuppc /data/backuppc/bin/BackupPC_xls_report
150
151	=head1 Additional requirements
152
153	You will also need several other command-line utilities to make
154	C<BackupPC_burnArchiveCLI> run. Those include:
155
156	=over 4
157
158	=item mkisofs
159
160	=item cdrecord
161
162	Or equivavalent utility, on Debian you might want to use C<dvdrecord>.
163
164	=item eject
165
166	Please make a symlink from C</dev/cdrom> to you CD/DVD burner.
167
168	=item split
169
170	=back
171
172	=head1 Troubleshoting
173
174	=head2 md5sum problems on Debian
175
176	If you are running Debian stable and expiriencing error message like
177
178	-: No such file or directory
179
180	your C<md5sum> comes from C<coreutils> (which doesn't know how to handle C<->
181	as input) as opposed to C<md5sum> which comes with C<textutils> (which does).
182
183	Fix is simple, correct md5sum is allready installed, you just have to do:
184
185	sudo cp /usr/bin/md5sum.textutils /usr/bin/md5sum
186
187	=head1 Related projects
188
189	BackupPC allready has archival host which might suit your needs better (and
190	it's quite easier to install).
191
192	=over 4
193
194	=item PostgreSQL
195
196	L<http://www.postgresql.org/>
197
198	=item HyperEstraier
199
200	L<http://hyperestraier.sourceforge.net/>
201
202	=back
203
204	=head1 Authors
205
206	Ivan Klaric C<< <iklaric@gmail.com> >>
207
208	Dobrica Pavlinusic C<< <dpavlin@rot13.org> >>
209
210	=head1 Licence
211
212	This extension is released under GPL licence, same as BackupPC.
213