Storing message data with dicts!


The problem: storing a dag node component in a way that makes it easily callable and persistent.

As I’ve been both refactoring/optimizing our core libraries as well as updating locinator I came across this old issue. There are several ways of doing this, some better than others. Just been wrapping up rewrite of our attribute function library. A part of that was rolling out our msgList concept from cgmMeta to being outside meta as well as expanding on that with datList(more on that another day).

Short version

If you don’t care about the details and just wanna see code, grab the last master branch build of our tools and you can find the main functions here:

  • cgm.core.lib.attribute_utils.set_message/get_message
  • Walkthrough example of datList/msgList with new stuff — cgm.core.examples.help_datList_msgList.py
  • Note — There may be a lot of script editor activity on the example stuff as I have DEBUG on in the module currently.

Long version

Let’s say we wanna store an object ‘null1’ to call and we’re storing on ‘storageNull’ How might we do that.

  • string attr – example: storageNull.stringAttr = null1
    • This works as long as there is only one object named ‘null1’ and as long as ‘null1’ is never renamed. So in short, it works rather poorly.
  • msgAttr – example storageNull.msgAttr >>connection>> null.msg
    • This works great and was my preferred method up to this point.

The conundrum on locinator was that I had some locator types that were created from a component say ‘geo.vtx[123]’ for example. My solution back in 2010ish when I wrote it was to just use a string for the whole thing and just hope there wasn’t a name conflict.

So, how might we store this in a persistent manner. Having learned a few things since back in twenty ought ten I said self, we can can better than that now.

The new implementation is as follows:

  1. We take our data to be stored and split out our base node from any component or attribute. Namely we split the first ‘.’ out and validate the bits to know what we have
  2. Store the main node as a standard message connection
  3. Store the extra bits to a json dict via Red9’s json string implementation. We also allow for a a specified dataAttr (our extra data attr) and dataKey (for the dict) for specific storage

So in this case our ‘geo.vtx[123]’ is split to the following:

  • storageNull.msgAttr >>connection>> geo.msg
  • sorageNull.dataAttr = {msgAttr/dataKey:vtx[123]}

We do this as a dict and not a simple string attr per stored object because we use lots of these and having two attrs for every stored message seemed overkill. Once I’d worked out the component store, attribute storing was pretty simple. If we wanted to also add ‘geo2.tx’, it would be added as:

  • storageNull.msgAttr2 >>connection>> geo2.msg
  • sorageNull.dataAttr = {msgAttr2/dataKey2:vtx[71], msgAttr/dataKey:tx}

The dataKey comes in particular use with our datList/msgList setup which is our solution to multi message attrs being rubbish for maintaining ordered data.

When the get_message call happens it first gets the msgAttr and then checks the default extra dat attr if none is specified. Whenever data is found it gets appended to the return.

Yes, you can do some of this stuff with objectSets or other avenues and sometimes those work great. This
is simply another way of storing data mainly for our rigging purposes.

Still refining this but happy so far. Thus ends this post.

 

Be Sociable, Share!

Comments are closed.


 
Creative Commons License

Categories