Search Queries and Filters with ElasticsearchCRUD

This article demonstrates how to do searches with ElasticsearchCRUD. The API provides a search model which can be used with the search API from Elasticsearch. You can search both sync/async and also provide you own JSON string or use the search model.

Lots of examples can be found here:
https://github.com/damienbod/ElasticsearchCRUD/tree/master/src/ElasticsearchCRUDNUnit.Integration.Test

Other Tutorials:

Part 1: ElasticsearchCRUD introduction
Part 2: MVC application search with simple documents using autocomplete, jQuery and jTable
Part 3: MVC Elasticsearch CRUD with nested documents
Part 4: Data Transfer from MS SQL Server using Entity Framework to Elasticsearch
Part 5: MVC Elasticsearch with child, parent documents
Part 6: MVC application with Entity Framework and Elasticsearch
Part 7: Live Reindex in Elasticsearch
Part 8: CSV export using Elasticsearch and Web API
Part 9: Elasticsearch Parent, Child, Grandchild Documents and Routing
Part 10: Elasticsearch Type mappings with ElasticsearchCRUD
Part 11: Elasticsearch Synonym Analyzer using ElasticsearchCRUD
Part 12: Using Elasticsearch German Analyzer
Part 13: MVC google maps search using Elasticsearch
Part 14: Search Queries and Filters with ElasticsearchCRUD
Part 15: Elasticsearch Bulk Insert
Part 16: Elasticsearch Aggregations With ElasticsearchCRUD
Part 17: Searching Multiple Indices and Types in Elasticsearch
Part 18: MVC searching with Elasticsearch Highlighting
Part 19: Index Warmers with ElasticsearchCRUD

Getting started:

ElasticsearchCRUD provides a number or search possibilities. This post shows different examples which should help in understanding or make it easier to use the search model parameter. When the model has be configured and created, it can be sent as a HTTP request post with the following methods:

public ResultDetails<SearchResult<T>> Search<T>(
   Search search, 
    SearchUrlParameters searchUrlParameters = null
)

//or 

public async Task<ResultDetails<SearchResult<T>>> SearchAsync<T>(
   string searchJsonParameters, 
   SearchUrlParameters searchUrlParameters = null
)

The search model can send queries with filters, queries or both. Here’s an example of a simple MatchAllFilter:

var search = new Search 
{ 
    Filter = new Filter(
        new MatchAllFilter()
    )
};

The Search model contains a Filter property and a Query property which can contain any query or any filter type. The required objects can be chosen as required.
Here’s an example of a simple PrefixQuery:

var search = new Search
{
    Query = new Query(new PrefixQuery("name", "on"))
};

Or a more complex search query could look like this:

var search = new Search
{
	Query = new Query(
		new Filtered( 
			new Filter( 
				new MatchAllFilter { Boost = 1.1 } 
			)
		) 
		{ 
			Query = new Query(new MatchAllQuery())
		}		
	)
};

This above search would be sent to Elasticsearch as follows:

POST http://localhost:9200/<index>/<type>/_search HTTP/1.1
Content-Type: application/json
Host: localhost:9200
Content-Length: 86
Expect: 100-continue

{ 
  "query":{
   "filtered":{
      "query":{"match_all":{}},
      "filter":{"match_all":{"boost":1.1}}
    }
  }
}

You can also use the ToString() method of the Search class to display the JSON query which will be sent to Elasticsearch. This is useful when debugging queries or filters.

An example of a GeoShapeQuery:

var search = new Search
{
	Query = new Query(
			new GeoShapeQuery("circletest",
				new GeoShapePolygon
				{
					Coordinates = new List<List<GeoPoint>>
					{
						new List<GeoPoint>
						{
							new GeoPoint(40,40),
							new GeoPoint(50,40),
							new GeoPoint(50,50),
							new GeoPoint(40,50),
							new GeoPoint(40,40)
						}
					}
				}
			)
	)
};

gets translated to:

{ 
  "query":{ 
     "geo_shape":{
       "circletest":{ 
          "shape":{ 
            "type":"polygon",
            "coordinates":[[
              [40.0,40.0],
              [50.0,40.0],
              [50.0,50.0],
              [40.0,50.0],
              [40.0,40.0]
            ]]
          }
        }
     }
  }
}

A FunctionScoreQuery with a Linear Decay function could be configured like this:

var search = new Search
{
	Query = new Query(
		new FunctionScoreQuery(
			new MatchAllQuery(),
			new List<BaseScoreFunction>
			{
				new LinearGeoPointFunction(
				  "location", 
				  new GeoPoint(40,40), 
				  new DistanceUnitKilometer(100) 
				)
				{
					Decay=0.3,
					Filter = new MatchAllFilter(),
					Offset= 3,
					Weight= 3.0
				}
			}
		)
	)
};

Sorting

The search model also supports sorting. Normal sorting, GeoPoint sorting or script sorting is supported.

Here’s an example of a Geo sort:

var search = new Search
{
	Query = new Query(new MatchAllQuery()),
	Sort = new SortHolder(
		new List<ISort>
		{
			new SortGeoDistance("location", DistanceUnitEnum.km)
			{
				GeoPoints = new List<GeoPoint>
				{
					new GeoPoint(46, 46),
					new GeoPoint(49, 46),
				},
				Order=OrderEnum.asc, Mode = SortModeGeo.max				
			}
		}
	)
};

Or another example of a script sort:

var search = new Search
{
	Query = new Query(new MatchAllQuery()),
	Sort = new SortScript("doc['lift'].value * factor")
	{
		Order = OrderEnum.asc,
		ScriptType= "number",
		Params = new List<ScriptParameter>
		{
			new ScriptParameter("factor", 1.5)
		}
	}
};

At the time of writing, all the possible configurations for the different queries and filters should be supported. Some of the Elasticsearch documentation is not actual and some of the documented features don’t work. All of the search model classes have integration tests which use a real Elasticsearch search engine (elasticsearch-1.4.2). This will help in keeping the code actual.

Links:

https://github.com/damienbod/ElasticsearchCRUD/tree/master/ElasticsearchCRUD.Integration.Test/SearchTests

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html

Tags: .NET, Elasticsearch, ElasticsearchCRUD, filter, Query, search