Data import
===========

SSH into the EU or US gateway:

.. code-block:: bash

    ssh -l root 54.229.115.35  # EU
    ssh -l root 107.21.39.107  # US

Once connected, go to the yellow-search project folder.

.. code-block:: bash

    cd /opt/yellow-search

Export data from production
---------------------------

Setup the environment to access the slave MySQL server (called demeter in both
zones).

.. code-block:: bash

    export OPENDI_MYSQL_HOST=demeter
    export OPENDI_MYSQL_USER=exporter
    export OPENDI_MYSQL_PASS=...

Export data from MySQL to a format which can be imported into Solr.

.. code-block:: bash

    bin/export categories branchenbuchCH CH
    bin/export listings --threshold=5000000 branchenbuchCH CH

This will export the data to the ``var/export`` folder.

Import data into Solr
---------------------

Set up your environment to access the production server in the chosen zone:

.. code-block:: bash

    export OPENDI_SOLR_URL=http://internal-solr-eu-208802902.eu-west-1.elb.amazonaws.com:8983/solr/
    export OPENDI_SOLR_URL=http://internal-solr-us-505242482.us-east-1.elb.amazonaws.com:8983/solr/

If desired, first delete existing data. This is not always required since
new data will overwrite existing if the ID matches.

.. code-block:: bash

    vendor/bin/solr delete categories
    vendor/bin/solr delete listings

Import the data generated above.

.. code-block:: bash

    vendor/bin/solr import categories var/export/categories-ch.json
    vendor/bin/solr import listings var/export/listings-ch*.json

It's recommended to optimize the data after the import.

.. code-block:: bash

    vendor/bin/solr optimize categories
    vendor/bin/solr optimize listings