← Newer Index Older →

The evil JSON packaging problem Sat, Aug 5 2017 PM

{
    "evil" : true
}

There is software floating about the opensource world from JSON.org thats licensed under MIT, a well understood opensource licence, but comes with an extra clause "The Software shall be used for Good, not Evil."

This is usually a non issue which can be handled by your local shaman or witchdoctor. But certain organizations like debian find the evil clause legally ambiguous, and they will simply not package software using this license. Read more here: https://wiki.debian.org/qa.debian.org/jsonevil

The problem is that the core Pothos library and several toolkits rely on Poco's JSON library, which contains this license clause. Debian and others can package Poco, but not its JSON component. And so therefore, Pothos can't be packaged as long as it has this dependency. I would like to see Pothos get packaged into more distros, so we need to fix this.

Now its time for some real evil....!

Whats a scheduling framework using JSON for?

JSON is just a well known, well supported text format for storing numbers, strings, arrays, and dictionaries. Pothos libraries and toolkits use JSON for:

Topology stats for performance monitoring
Exporting a topology to a list of blocks and connections
Building a topology from a list of blocks and connections
Unit test plans for feeder source and collector sink
Arbitrary device information (SDR, IIO, Audio toolkits)

A replacement for Poco::JSON

For starter's, I don't like the idea of adding another core dependency, but doing so may just be the lesser "evil"...

Such a dependency needs to be a C++ library, cross platform, easy to build, and hopefully available on many distros. And JSON is so pervasive, that there are actually too many options to wade through. But nevertheless, I have chosen this JSON implementation written by Niels Lohmann.

Some pros about using json.hpp from nlohmann

Its a single header file and no library. There's nothing to compile, simply include the header. If the platform doesn't have json.hpp installed, the cmake script simply downloads it from github.
The API is very clean and intuitive. It makes good use of C++11 features, and saves lines of code compared to Poco::JSON
There is already a package in ubuntu
And a homebrew recipe: brew tap nlohmann/json && brew install nlohmann_json

Some cons about using json.hpp from nlohmann

Its bleeding edge in terms of compiler requirements. That means requiring GCC 5 and MSVC 2015 and up for full C++11 compliance.
I have to refactor working Poco::JSON code to use the new API.
And must change a few APIs that used Poco::JSON classes in the interface.

I'm not thrilled about bumping compiler requirements, but at the same time its not really a big deal. For some perspective, this isn't yet in a release and Pothos 0.5 will still be maintained for some time regardless. Also we have two versions of visual studio (2015 and 2017) that can build Pothos, and Ubuntu LTS 16.04 comes with a compatible GCC 5 release. Even the previous Ubuntu LTS 14.04 can be upgraded to use GCC 5.

Known library and API changes

The master branch (for Pothos 0.6 and up) requires this json.hpp header, which can be downloaded automatically by the build, so most will not even notice.
Device info entries are now serialized JSON strings and not Poco::JSON::Objects. This effected the core library, and audio, SDR, and IIO toolkits
The feeder source block now uses serialized JSON strings for its test plan. This effected unit testing in blocks, comms, python, audio, SDR, gr-pothos, IIO, and opencl toolkits. Some will have to update their unit tests if they were using the feeder source block for testing. In fact, Poco::JSON can still be used, outside of main toolkits, you just have to serialize the result.
The master branches of the respective toolkits now require Pothos 0.6 to match the change in compatibility