Solr comes with some sample data found at example/exampledocs. We saw this data loaded as part of creating the techproducts Solr core when we started Solr. We're going to use that for the remainder of this chapter so that we can explore Solr more, without getting into schema design and deeper data loading options. For the rest of the book, we'll base the examples on the digital supplement to the book—more on that later.

We're going to re-index the example data by using the post.jar Java program, officially called SimplePostTool. Most JAR files aren't executable, but this one is. This simple program takes a Java system variable to specify the collection: -Dc=techproducts, iterates over a list of Solr-formatted XML input files, and HTTP posts it to Solr running on the current machine —http://localhost:8983/solr/techproducts/update. Finally, it will send a commit command, which will cause documents that were posted prior to the commit to be saved and made visible. Obviously, Solr must be running for this to work. Here is the command and its output:

>> cd example/exampledocs
>> java –Dc=techproducts -jar post.jar *.xml
SimplePostTool version 5.0.0
Posting files to [base] url http://localhost:8983/solr/techproducts/update using content-type application/xml...
POSTing file gb18030-example.xml
POSTing file hd.xml
etc.
14 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/techproducts/update...

If you are using a Unix-like environment, you have an alternate option of using the /bin/post shell script, which wraps the SimplePostTool.

Let's take a look at one of these XML files we just posted to Solr, monitor.xml:

<add>
  <doc>
    <field name="id">3007WFP</field>
    <field name="name">Dell Widescreen UltraSharp 3007WFP</field>
    <field name="manu">Dell, Inc.</field>
    <!-- Join -->
    <field name="manu_id_s">dell</field>
    <field name="cat">electronics</field>
    <field name="cat">monitor</field>
    <field name="features">30" TFT active matrix LCD, 2560 x 1600, .25mm dot pitch, 700:1 contrast</field>
    <field name="includes">USB cable</field>
    <field name="weight">401.6</field>
    <field name="price">2199</field>
    <field name="popularity">6</field>
    <field name="inStock">true</field>
    <!-- Buffalo store -->
    <field name="store">43.17614,-90.57341</field>
  </doc>
</add>

The XML schema for files that can be posted to Solr is very simple. This file doesn't demonstrate all of the elements and attributes, but it shows the essentials. Multiple documents, represented by the <doc> tag, can be present in series within the <add> tag, which is recommended for bulk data loading scenarios. This subset may very well be all that you use. More about these options and other data loading choices will be discussed in Chapter 4, Indexing Data.

Point your browser to http://localhost:8983/solr/#/techproducts/query—this is the query form described in the previous section. The search box is labeled q. This form is a standard HTML form, albeit enhanced by JavaScript. When the form is submitted, the form inputs become URL parameters to an HTTP GET request to Solr. That URL and Solr's search response is displayed to the right. It is convenient to use the form as a starting point for developing a search, but then subsequently refine the URL directly in the browser instead of returning to the form.

Run a query by replacing the *:* in the q field with the word lcd, then clicking on the Execute Query button. At the top of the main content area, you will see a URL like this http://localhost:8983/solr/techproducts/select?q=monitor&wt=json&indent=true. The URL specifies that you want to query for the word lcd, and that the output should be in indented JSON format.

Below this URL, you will see the search result; this result is the response of that URL.

By default, Solr responds in XML, however the query interface specifies JSON by default. Most modern browsers, such as Firefox, provide a good JSON view with syntax coloring and hierarchical controls. All response formats have the same basic structure as the JSON you're about to see. More information on these formats can be found in Chapter 4, Indexing Data.

The JSON response consists of a two main elements: responseHeader and response. Here is what the header element looks like:

"responseHeader": {
    "status": 0,
    "QTime": 1,
    "params": {
      "q": "lcd",
      "indent": "true",
      "wt": "json"
    }
  }
…

The following are the elements from the preceding code snippet:

Next up is the most important part, the results:

"response": {
    "numFound": 5,
    "start": 0,

The numFound value is the number of documents matching the query in the entire index. The start parameter is the index offset into those matching (ordered) documents that are returned in the response below.

Often, you'll want to see the score of each matching document. The document score is a number that represents how relevant the document is to the search query. This search response doesn't refer to scores because it needs to be explicitly requested in the fl parameter—a comma-separated field list. A search that requests the score via fl=*,score will have a maxScore attribute in the "response" element, which is the maximum score of all documents that matched the search. It's independent of the sort order or result paging parameters.

The content of the result element is a list of documents that matched the query. The default sort is by descending score. Later, we'll do some sorting by specified fields.

{
        "id": "9885A004",
        "name": "Canon PowerShot SD500",
        "manu": "Canon Inc.",
        "manu_id_s": "canon",
        "cat": [
          "electronics",
          "camera"
        ],
        "features": [
          "3x zoop, 7.1 megapixel Digital ELPH",
          "movie clips up to 640x480 @30 fps",
          "2.0\" TFT LCD, 118,000 pixels",
          "built in flash, red-eye reduction"
        ],
        "includes": "32MB SD card, USB cable, AV cable, battery",
        "weight": 6.4,
        "price": 329.95,
        "price_c": "329.95,USD",
        "popularity": 7,
        "inStock": true,
        "manufacturedate_dt": "2006-02-13T15:26:37Z",
        "store": "45.19614,-93.90341",
        "_version_": 1500358264225792000
      },
...

The document list is pretty straightforward. By default, Solr will list all of the stored fields. Not all of the fields are necessarily stored—that is, you can query on them but not retrieve their value—an optimization choice. Notice that it uses the basic data types of strings, integers, floats, and Booleans. Also note that certain fields, such as features and cat are multivalued, as indicated by the use of [] to denote an array in JSON.

This was a basic keyword search. As you start using more search features such as faceting and highlighting, you will see additional information following the response element.

Let's take a look at the statistics available via the Plugins / Stats page. This page provides details on all the components of Solr. Browse to CORE and then pick a Searcher. Before we loaded data into Solr, this page reported that numDocs was 0, but now it should be 32.

Now take a look at the update handler stats by clicking on the UPDATEHANDLER and then expand the stats for the update handler by clicking on the updateHandler toggle link on the right-hand side of the screen. Notice that the /update request handler has some stats too:

If you think of Solr as a RESTful server, then the various public end points are exposed under the QUERYHANDLER menu. Solr isn't exactly REST-based, but it is very similar. Look at the /update to see the indexing performance, and /select for query performance.

The final destination of our quick Solr tour is to visit the so-called browse interface—available at http://localhost:8983/solr/techproducts/browse. It's for demonstrating various Solr features:

This is also a demonstration of Solritas, which formats Solr requests using templates that are based on Apache Velocity. The templates are VM files in example/techproducts/solr/techproducts/conf/velocity. Solritas is primarily for search UI prototyping. It is not recommended for building anything substantial. See Chapter 9, Integrating Solr, for more information.

Here is a screenshot of the browse interface; not all of it is captured in this image: