Shadowserver integration updated

There is a lot of development going on, but not everything is directly visible. So we'll not bore you with details and stick to the useable stuff.

We recently added integration with Shadowserver.org. Up until now, their data was only used for discovery. For multi-tenant users, each organization had to enter the API keys separately resulting in extra work for users and more load on our servers.

This update allows multi-tenant users like MSSPs, Hosters and Network Operators to add the Shadowserver integration on group level. ShadowTrack will check daily, get all assets and events available in Shadowserver, and map these to the organizations you have in your group.

The new integration also uses the Shadowserver data much better. Besides discovery, you can import the device_id data (both IPv4 and IPv6) in to a special index in ShadowTrackr called shadowserver_device_id. The Shadowserver reports are now also parsed and processed as events. So, if one of your servers is connecting to a sinkhole or honeypot the alert for that will show up in your ShadowTrackr events.

Updated Magic queries documentation

As part of a continuing effort to update and improve documentation, the first information on magic queries is now available.

Sometimes you want data from ShadowTrackr that you know is in there, but cannot get out with the query language. A good example is if you want to combine data from two or more indexes. The query language does not support joins. This is where magic queries are used.

All magic queries start with a $. There are a number of existing ones, and they are now listed in the documentation. For quick access, just type $ in the search bar in the gui and auto-complete will show you what's available.

If you cannot find the magic query that you need, contact support and we'll try to make a new magic query for your specific needs.

Updated scheduling algorithms

Most scheduling algorithms have been reviewed and updated. The trigger for this was that sometimes an item could drop out or get stuck somewhere and was not properly scanned for a while. That is fixed now, and the resulting backlog of unscanned items was processed over the weekend.