Buffer Length Checking on write_json() ?

[mispriced]

I see where this happens on resizable buffers, but I don't see any option to pass the length of a fixed char[] buffer to the write methods. Ideally, we'd fail with an error if the generated JSON doesn't fit in the buffer.

Also, any thoughts on ever supporting something like JSON pointer syntax for writing?

[stephenberry]

The aim for a fixed char buffer was to allow optimum performance by not checking the output length, given that the generated size was known to be within the allocated buffer. Adding checks will increase the runtime cost, but I do see where it could be useful in embedded environments where you don't want to have any dynamic allocations.

What is your use case for this feature? I do think it would be good to add, I just want to take into account use cases in developing a solution.

As for JSON pointer syntax for writing, Glaze does support partial writing with JSON pointer syntax: Partial Write
This only supports compile time JSON pointers. Are you looking for runtime JSON pointer writing? I do think this would be a good feature to add as well.

I will open issues for each of these requested features. Thanks for the feedback!

[mispriced]

My use case is that I publish a message with nearly identical structure very frequently. I'd prefer to not dynamically allocate and just re-use the same output buffer. I don't perfectly control the size of the messages as some field sizes might be more digits or less per update etc. My design was just to use a "pretty big char[]" but I'd rather catch an error and not publish a message (or fall back to dynamic allocation there) than use a stupidly big buffer.

Run time partial writing would be incredible. Ideally, I'd just update the contents of various fields that changed per update. I'm betting this is pretty tricky implementation wise (I'm guessing some sort of pending buffer so we can apply multiple changes and then move things around accordingly)

[stephenberry]

You can reuse the same output buffer with a std::string by calling glz::write_json(obj, buffer). As long as buffer is kept alive, Glaze will reuse this memory every time you call write_json. This allows you to minimize memory usage (not needing a stupidly big buffer), but keeps performance high by reusing memory. If you don't have anywhere to put your buffer (very unlikely), then you can always make it a static threadlocal.

Run time partial writing will come at some performance cost, but it won't be too hard to implement because we already support runtime partial reading.

[mispriced]

I think I end up in a similar situation using a string as the backing buffer -- eventually a big message comes along triggering a resize and I'm going to have a bigger chunk of allocated space. Could certainly manage and resize the buffer back down if that were to occur, but there's a bit less control with std::string than just an array/vector of char I think.

[stephenberry]

Glaze will handle a std::vector in the same way as a std::string. If you don't want allocations as the buffer grows then you can use the .data() method on either the vector or string to just write to the underlying allocated memory without growing.

Both std::vector and std::string have shrink_to_fit methods that you can use to reduce the memory to the capacity that is actually required after you call write_json. However, you need to test that these actually work on your platform/compiler, as some low level systems will not actually do anything because of how memory is managed.

[stephenberry]

I added the issues #1285 and #1286 for buffer length write checking and runtime JSON pointer syntax for writing.

[mispriced]

thanks you're the best!