JarCastingImportant Note
As of version 1.3.2, the preferred way to specify nested filters is to use square brackets intead of braces.
Preferred: assignee[firstName]
No longer Preferred but will still work: assignee{firstName}
The reason for this is that newer versions of Tomcat no longer allow braces to be specified on the url without being escaped. Square brackets are still permitted in the url and it is preferred to make the syntax url friendly.
Squiggly Filter For Jackson
Contents
- What is it?
- Prerequisites
- Installation
- General Usage
- Reference Object
- Top-Level Filters
- Nested Filters
- Dot Syntax
- Regex Filters
- Other Filters
- Resolving Conflicts
- Excluding Fields
- Property Views
- More Examples
- Custom Integration
- Changing the Defaults
- Metrics
- Limitations
What is it?
The Squiggly Filter is a Jackson JSON PropertyFilter, which selects properties of an object/list/map using a subset of the Facebook Graph API filtering syntax.
Probably the most common use of this library is to filter fields on the querystring like so:
?fields=id,reporter[firstName]
Integrating Squiggly into your webapp is covered in Custom Integration.
Requirements
- Java 7+
- ANTLR
- Commons Lang 3
- Google Guava
- Jackson JSON (version 2.6+)
Installation
Maven
<dependency>
<groupId>com.github.bohnman</groupId>
<artifactId>squiggly-filter-jackson</artifactId>
<version>1.3.18</version>
</dependency>
General Usage
ObjectMapper objectMapper = Squiggly.init(new ObjectMapper(), "assignee{firstName}");
Issue object = new Issue(); // replace this with your object/collection/map here
System.out.println(SquigglyUtils.stringify(objectMapper, object));
Alternatively, if you need more control over configuring the ObjectMapper, you can do it this way:
String filterId = SquigglyPropertyFilter.FILTER_ID;
SquigglyPropertyFilter propertyFilter = new SquigglyPropertyFilter("assignee[firstName]"); // replace with your filter here
SimpleFilterProvider filterProvider = new SimpleFilterProvider().addFilter(filterId, propertyFilter);
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setFilterProvider(filterProvider);
objectMapper.addMixIn(Object.class, SquigglyPropertyFilterMixin.class);
Issue object = new Issue(); // replace this with your object/collection/map here
System.out.println(SquigglyUtils.stringify(objectMapper, object));
Also, you can generate a Plain Old Java Object (POJO) instead of a JSON String
ObjectMapper objectMapper = Squiggly.init(new ObjectMapper(), "assignee{firstName}");
System.out.println(SquigglyUtils.objectify(objectMapper, object, Object.class));
For applying filter on Collection of Objects and for returning Collection of POJOs instead of JSON String
List<User> users = Arrays.asList(
new User("Peter", 12, "Dinklage"),
new User("Lena", 13, "Heady"));
String filter = "firstName,age";
ObjectMapper objectMapper = Squiggly.init(new ObjectMapper(), filter);
List<User> filteredUsers = SquigglyUtils.listify(objectMapper, users, User.class);
// setify is also availble
Reference Object
For the filtering examples, let's use an the example object of type Issue
{
"id": "ISSUE-1",
"issueSummary": "Dragons Need Fed",
"issueDetails": "I need my dragons fed pronto.",
"reporter": {
"firstName": "Daenerys",
"lastName": "Targaryen"
},
"assignee": {
"firstName": "Jorah",
"lastName": "Mormont"
},
"actions": [
{
"id": null,
"type": "COMMENT",
"text": "I'm going to let Daario get this one.",
"user": {
"firstName": "Jorah",
"lastName": "Mormont"
}
},
{
"id": null,
"type": "CLOSE",
"text": "All set.",
"user": {
"firstName": "Daario",
"lastName": "Naharis"
}
}
],
"properties": {
"priority": "1",
"email": "[email protected]"
}
}
Top-Level Filters
Select No Fields
String filter = "";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {}
Select Single Field
filter = "id";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"id":"ISSUE-1"}
Select Multiple Fields
filter = "id,issueSummary"
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"id":"ISSUE-1", "issueSummary":"Dragons Need Fed"}
Select Fields Using Wildcards
filter = "issue*";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed", "issueDetails": "I need my dragons fed pronto."}
Select All Fields
filter = "**";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints the same json as our example object
Select All Fields of object, but only base fields of associated objects (more on this later)
filter = "*";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
/* prints the following:
{
"id": "ISSUE-1",
"issueSummary": "Dragons Need Fed",
"issueDetails": "I need my dragons fed pronto.",
"reporter": {
"firstName": "Daenerys",
"lastName": "Targaryen"
},
"assignee": {
"firstName": "Jorah",
"lastName": "Mormont"
},
"actions": [
{
"id": null,
"type": "COMMENT",
"text": "I'm going to let Daario get this one.."
},
{
"id": null,
"type": "CLOSE",
"text": "All set."
}
],
"properties": {
"priority": "1",
"email": "[email protected]"
}
}
*/
Nested Filters
Select Single Nested Field
String filter = "assignee[firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"assignee":{"firstName":"Jorah"}}
Select Multiple Nested Fields
String filter = "actions[text,type]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"actions":[{"type":"COMMENT","text":"I'm going to let Daario get this one.."},{"type":"CLOSE","text":"All set."}]}
// NOTE: use can also use wildcards (e.g. actions{t*})
Select Same Field From Different Nested Objects
String filter = "(assignee,reporter)[firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"reporter":{"firstName":"Daenerys"},"assignee":{"firstName":"Jorah"}}
Select Deeply Nested Field
String filter = "actions[user[lastName]]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"actions":[{"user":{"lastName":"Mormont"}},{"user":{"lastName":"Naharis"}}]}
Dot Syntax
As an alternative to using the braces syntax for nested filter, you can use the dot syntax
String filter = "assignee.firstName";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"assignee":{"firstName":"Jorah"}}
You can exclude fields using the dot syntax. Note that the exclusion applies to the last field.
String filter = "-assignee.firstName";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"assignee":{"lastName":"Mormont"}}
You can also combine the dot syntax with the nested syntax.
String filter = "actions.user[firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"actions":[{"user":{"firstName":"Jorah"}},{"user":{"firstName":"Daario"}}]}
One limitation is that you cannot use the | syntax with the dot syntax
String filter = "(actions.user,assignee)[firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// throws exception
Regex Filters
In addition to using wildcards, you can also use regular expressions.
Here's an example:
String filter = "~iss[a-z]e.*~";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed","issueDetails":"I need my dragons fed pronto."}
Notice the tildes mark the begin and of the regex pattern.
You can also specifiy a case insensitive match.
String filter = "~iss[a-z]esumm.*~i";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed"}
Why use tildes and not forward slashes for regular expressions? Tildes are query string friendly and forward slashes are not.
However, you may use forward slashes if you like.
String filter = "/iss[a-z]esumm.*/i";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed"}
Other Filters
Selecting from Maps
Selecting from maps is the same as selecting from objects. Instead of selecting from fields, you are selecting from keys. The main downside of selecting from maps that their matches are unable to be cached.
Map<String, Object> map = new HashMap<>();
map.put("foo", "bar");
map.put("bear", "baz");
String filter = "foo";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, map));
// prints {"foo":"bar"}
Selecting from Collections (Lists/Arrays/Etc).
Selecting from collection just assumes the top-level objects are the elements in the collection, not the collection itself.
List<User> list = Arrays.asList(
new User("Peter", "Dinklage"),
new User("Lena", "Heady")
);
String filter = "firstName";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, list));
// prints [{"firstName":"Peter"}, {"firstName":"Lena"}]
Resolving Conflicts
When a filter includes two criteria that match the same field, the one that is more specific wins.
For example, if the filter is "**,reporter[firstName]", then all fields will be excluded. However, the reporter field will only include the firstName field.
Specificity is determined using the following logic:
- an exact name is the most specific
- a ** is the least specific
- a * is the second to least specific
- otherwise, the number of non-wildcard characters is counted, the higher the number, the more specific
- if two filters have the same specificity, the latter one is chosen
Excluding Fields
In order to exclude fields, you need to prefix the field name with a minus sign (-).
Let's look at an example
String filter = "-reporter";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints everything except the reporter field
Here's an example excluding a nested field:
String filter = "**,reporter[-firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints everything, the reporter object will only have the firstName excluded
NOTE: Excluded fields can't have nested filters
String filter = "**,-reporter[firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// throws an exception
Property Views
In addition to selecting fields by name, you can assign a name to a group of fields. This is called a property view.
Reference Objects
Let's use these reference objects for the examples.
@Target(FIELD)
@Retention(RUNTIME)
@Documented
@PropertyView({"super"})
public @interface SuperView {
}
class Address {
String line1 = "55 Hollywood Blvd.";
String line2 = "";
String city = "Hollywood";
String state = "CA";
@SuperView
double lat;
@SuperView
double lon;
}
class User {
String firstName = "Peter";
String lastName = "Dinklage";
@PropertyView("secret")
String phone = "555-555-1212";
@SuperView
Address address;
}
The Base View
If nothing is annotated on a field, it is assumed to belong to the "base" view. There is @BaseView convenience annotation, but it's not needed. See Changing Defaults to alter this behavior.
In the case of User, fields firstName and lastName belong to the "base" view.
String filter = "base";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, user));
// prints {"firstName":"Peter","lastName","Dinklage"}
Using the @PropertyView Annotation
If you look at the phone field of the User class, you'll notice the @PropertyView("secret") annotation on the phone field. This indicates that the phone field belongs to the "secret" view.
String filter = "secret";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, user));
// prints {"firstName":"Peter","lastName","Dinklage", "phone":"555-555-1212"}
Wait a minute! Why was the firstName and lastName field included? Even though we specified a certain view, the base fields are always included. See Changing Defaults to alter this behavior.
Note that you can also specifiy multiple views in the annotation - @PropertyView({"one", "two", "three"}))
You can also specify @PropertyView on getters and setters.
Using a Derived Annotation
If you look at the address field of the User class, you'll notice the @SuperView annotation. Looking at the @SuperView declaration, you'll notice it is annotated with a @PropertyView("super"). This is how you create a derived annotation.
Let's try it out.
String filter = "super";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, user));
// prints {"firstName":"Peter","lastName","Dinklage", "address":{"line1":"55 Hollywood Blvd.","line2":"","city":"Hollywood","state":"CA"}}
Wait another minute! The Address class has @SuperView annotations as well. Why weren't they include? Well, the view only applies to the current level. In order to get the super views of the address, you would have to specifiy a filter "super[super]". See Changing Defaults to alter this behavior.
More Examples
There are more examples in the test directory.
Custom Integration
Imagine you are building a webapp where you want to specify the fields on the querystring.
E.g. `/some/path?fields=a,b{c} ``
You'll notice in all of our examples, we passed in a filter expression that never changes. This doesn't work well for the case of specifying filters on a querystring.
Enter the SquigglyContextProvider. This interface allows you to customize how to retrieve the fields.
The RequestSquigglyContextProvider
All servlet-based integrations use the RequestSquigglyContextProvider, which has the general initialization in the form of:
Squiggly.init(objectMapper, new RequestSquigglyContextProvider());
Automatically wrapping fields with an outer filter
Let's say you have the following class called Page that looks like this:
public class Page<T> {
private final int pageNumber;
private final int pageSize;
private final List<T> items;
public ListResponse(List<T> items, int pageNumber, int pageSize) {
this.items = checkNotNull(items);
this.pageNumber = pageNumber;
this.pageSize = pageSize;
}
public List<T> getItems() {
return items;
}
public int getPageNumber() {
return pageNumber;
}
public int getPageSize() {
return pageSize;
}
}
Let's say you have an endpoint called /issues that looks like this:
public Page<Issue> findIssues(String query, int pageNumber, int pageSize) {
List<Issue> issues = issueService.findIssues(query, pageNumber, pageSize);
return new Page<Issue>(issues, pageNumber, pageSize);
}
In order to get specify the issue property, you now have to wrap all filters with items[] like so:
GET /issues?fields=items{id}&query=some-query&&pageNumber=1&pageSize=10
This is kind of annoying. Fortunately, we can avoid this inconvenience by using a hook method in RequestSquigglyContextProvider.
Here's how it would look:
Squiggly.init(objectMapper, new RequestSquigglyContextProvider() {
@Override
protected String customizeFilter(String filter, HttpServletRequest request, Class beanClass) {
if (filter != null && Page.class.isAssignableFrom(beanClass)) {
filter = "items[" + filter + "]";
}
return filter;
}
});
Now you can do this:
GET /issues?fields=id&query=some-query&&pageNumber=1&pageSize=10
Generic Servlet Webapp
You can find an example of using Squiggly Filter in a webapp under the examples/servlet directory.
Spring Boot Web Application
You can find an example of using Squiggly Filter in Spring Boot under the examples/spring-boot directory.
Dropwizard
You can find an example of using Squiggly Filter in Dropwizard under the examples/dropwizard directory.
Changing Defaults
You have the ability to customize Squiggly by creating a file called squiggly.properties in the root of the classpath.
Cache Config
The following properties are used to control various caches in Squiggly Filter. Internally, these properties get converted to a Guava CacheBuilderSpec. Please refer to to the documentation to see all the values that are available.
- parser.nodeCache.spec=maximumSize=10000
- filter.pathCache.spec=maximumSize=10000
- property.descriptorCache.spec=<empty>
Enable/Disable adding non-annotated fields to the "base" view
- property.addNonAnnotatedFieldsToBaseView=true
Enable/Disable inclusion of base fields for nested objects
- filter.implicitlyIncludeBaseFields=true
Enable/Disable inclusion of base fields when a view is specified
- filter.implicitlyIncludeBaseFieldsInView=true
When set to false, base fields are not included when specifying a view
Enable/Disable View Propagation to Nested Filters
- filter.propagateViewToNestedFilters=false
When set to true, views are propagated to nested filters
Getting Config Info
Squiggly Filter provides 2 methods to get information about configuration.
SquigglyConfig.asMap() will return a map of the merged config that looks like the following:
{
"filter.implicitlyIncludeBaseFields": "true",
"filter.implicitlyIncludeBaseFieldsInView": "true",
"filter.pathCache.spec": "maximumSize=10000",
"filter.propagateViewToNestedFilters": "false",
"parser.nodeCache.spec": "maximumSize=10000",
"property.addNonAnnotatedFieldsToBaseView": "true",
"property.descriptorCache.spec": ""
}
SquigglyConfig.asSourceMap() will return a map of the config keys and paths where the entry was retrieved that looks like the following:
{
"filter.implicitlyIncludeBaseFields": "file:/path/one/squiggly.default.properties",
"filter.implicitlyIncludeBaseFieldsInView": "file:/path/one/squiggly.default.properties",
"filter.pathCache.spec": "file:/path/one/squiggly.default.properties",
"filter.propagateViewToNestedFilters": "file:/path/one/squiggly.default.properties",
"parser.nodeCache.spec": "file:/path/two/squiggly.properties",
"property.addNonAnnotatedFieldsToBaseView": "file:/path/two/squiggly.properties",
"property.descriptorCache.spec": "file:/path/two/squiggly.properties"
}
Metrics
Squiggly Filter provides an API for obtaining various metrics about the library, such as cache statistics. This allows users to monitor and adjust configuration as needed.
To use the metrics, you can do something the like following:
Map<String, Object> metrics = SquigglyMetrics.asMap();
System.out.println(SquigglyUtils.stringify(new ObjectMapper(), metrics));
This will print the following:
{
"squiggly.filter.pathCache.averageLoadPenalty": 0,
"squiggly.filter.pathCache.evictionCount": 0,
"squiggly.filter.pathCache.hitCount": 0,
"squiggly.filter.pathCache.hitRate": 1,
"squiggly.filter.pathCache.loadExceptionCount": 0,
"squiggly.filter.pathCache.loadExceptionRate": 0,
"squiggly.filter.pathCache.loadSuccessCount": 0,
"squiggly.filter.pathCache.missCount": 0,
"squiggly.filter.pathCache.missRate": 0,
"squiggly.filter.pathCache.requestCount": 0,
"squiggly.filter.pathCache.totalLoadTime": 0,
"squiggly.parser.nodeCache.averageLoadPenalty": 0,
"squiggly.parser.nodeCache.evictionCount": 0,
"squiggly.parser.nodeCache.hitCount": 0,
"squiggly.parser.nodeCache.hitRate": 1,
"squiggly.parser.nodeCache.loadExceptionCount": 0,
"squiggly.parser.nodeCache.loadExceptionRate": 0,
"squiggly.parser.nodeCache.loadSuccessCount": 0,
"squiggly.parser.nodeCache.missCount": 0,
"squiggly.parser.nodeCache.missRate": 0,
"squiggly.parser.nodeCache.requestCount": 0,
"squiggly.parser.nodeCache.totalLoadTime": 0,
"squiggly.property.descriptorCache.averageLoadPenalty": 0,
"squiggly.property.descriptorCache.evictionCount": 0,
"squiggly.property.descriptorCache.hitCount": 0,
"squiggly.property.descriptorCache.hitRate": 1,
"squiggly.property.descriptorCache.loadExceptionCount": 0,
"squiggly.property.descriptorCache.loadExceptionRate": 0,
"squiggly.property.descriptorCache.loadSuccessCount": 0,
"squiggly.property.descriptorCache.missCount": 0,
"squiggly.property.descriptorCache.missRate": 0,
"squiggly.property.descriptorCache.requestCount": 0,
"squiggly.property.descriptorCache.totalLoadTime": 0
}
Limitations
Using Serializers
If you use a custom serializer and write to a JsonGenerator directly, you will completely bypass Squiggly.
For example:
// Doesn't work with Squiggly
public class TestSerializer extends JsonSerializer<TestObject> {
@Override
public void serialize(TestObject value, JsonGenerator jgen, SerializerProvider provider) throws IOException {
jgen.writeStartObject();
jgen.writeStringField("a", value.getA());
jgen.writeStringField("c", value.getC());
jgen.writeEndObject();
}
}
Instead, you need to use the SerializerProvider, which will invoke Squiggly. The above serializer can be rewritten as:
// Works with Squiggly
public class TestSerializer extends JsonSerializer<TestObject> {
@Override
public void serialize(TestObject value, JsonGenerator jgen, SerializerProvider provider) throws IOException {
Map<String, Object> map = new HashMap<>();
map.put("a", value.getA());
map.put("c", value.getC());
provider.defaultSerializeValue(map, jgen);
}
}