Backup to HDFS while running cluster on local disk

Hrishikesh Gadre

2016-11-08 18:38:26 UTC

Hi Mike,

Thanks for bringing this up. You can certainly backup the index data stored
on local file-system to HDFS.

The HDFS backup repository implementation uses the same configuration
properties as expected by the HDFS directory factory. Here is the
description of the parameters,

- location (Optional) - This configuration parameter defines the default
location where the backups can be stored. If this parameter is not
configured, then you will need to explicitly specify the location parameter
to your backup and restore commands.
- solr.hdfs.home (Required) - This configuration parameter defines the
fully qualified URI for the root path of HDFS. e.g. hdfs://name-node-1/. In
case the index files are also stored on HDFS, this path refers to the
directory used to store index files in HDFS e.g. hdfs://name-node-1/solr
- solr.hdfs.confdir (Optional) - A directory (on local file-system)
which contains the configuration files for HDFS (e.g. hdfs-site.xml,
core-site.xml etc.)

I will also update the docs accordingly.

-Hrishikesh

Post by Mike Thomsen
We have SolrCloud running on bare metal but want the nightly snapshots to
be written to HDFS. Can someone give me some help on configuring the
HdfsBackupRepository?
<backup>
<repository name="hdfs" class=
"org.apache.solr.core.backup.repository.HdfsBackupRepository" default=
"false">
<str name="location">${solr.hdfs.default.backup.path}</str>
<str name="solr.hdfs.home">${solr.hdfs.home:}</str>
<str name="solr.hdfs.confdir">${solr.hdfs.confdir:}</str>
</repository>
</backup>
Not sure how to proceed on configuring this because the documentation is a
bit sparse on what some of those values mean in this context. The example
looked geared toward someone using HDFS both to store the index and do
backup/restore.
Thanks,
Mike