Information nesting 1
Lists, grids and tables
Last updated
Lists, grids and tables
Last updated
e objective of this session is to introduce the concept of data nesting. We will use the geometric construct of a grid to develop a visual analogy and build intuition.
So far we have developed some understanding of single values versus containers of multiple values i.e. lists. Our lists are equivalent in a sense to a single spreadsheet column of cells. So here will will look at how we can approach the subject of creating 2-dimensional data configurations with columns and rows i.e. tables.
The notion of ‘nesting’ allows us to construct more complex data shapes than single values and lists. If we treat a list of values as just another value itself, we can store a list inside another.
We begin with a regular series and note that the ledger numbers indicate the index of each value in the same fashion a spreadsheet’s rows are numbered (starting from one instead of zero). List values are presented in a vertical manner in exactly the same way we have in a spreadsheet’s column. Finally, we have a list header, i.e. “{0;0}” which we have not discussed. We can consider it as an equivalent of column letters used in spreadsheets.
Using the context menu on the series’ output we select the “graft” option item. This transforms the list into a table. The best way to interpret this behaviour is by using the spreadsheet analogy. We can observe that each entry is preceded with a column marker. Thus, instead of creating a column of values, we have a row spanning multiple columns.
The wire representation has also changed from a thick double line to a thick dotted line to highlight the difference in data shape. The figure below captures the relation between grafted data and the equivalent table form.
The “graft” operation can be also represented symbolically using nested brackets as seen below. With this representation it is easier to appreciate the concept of storing lists within lists also known as nesting.
Practice
Create a “series” component with “start” from 1 and count “5”, and connect its output to the “count” input of another “series”. Explain the result generated in a table sense?
Graft the output of a “series” and connect it to a “list length” component. Verify that the output is a table of ones.
Just for illustration purposes the number/letter construction is presented below in both the visual programming environment as well as the spreadsheet application. The panel component unfortunately presents a linearized version of the table where each column in stacked one after another.
For a geometric interpretation of how the Cartesian product works we may generate polylines directly from the output points. As you can visualize in the figure below this results into chaining across columns i.e. the “polyline” component consumes the points from the table column by column.
Practice
Use the “point list” display component to visualize the order of grid points created using the grafting logic above.
Exchange the two “series” in the figure above such that the Y-coordinate is linked to the grafted series. What is the result regarding the point order?
Connect a “series” into the X-coordinate of a point and connect the same “series” into the “count” of a new “series” before connecting the latter into the Y-coordinate of the same point.
The behaviour may feel peculiar at first but in fact it follows exactly the same logic we called “lacing” earlier. To see how this works we need to get into the details of this process step-by-step. Note that we need to focus on the shape and size of the data not the actual values. For this we will use the “concatenate” component which join character strings together. (Numbers are converted to text prior to concatenating)
[ ]1 ⮆ [ ]1 ⟶ [ ]1
Single value with single value produces a single value. This is straight forward and expected. Note that both inputs and outputs have the same shape and size.
[ ]n ⮆ [ ]n ⟹ [ ]n
Combining simple lists of the same size produces new lists with the same size as the input. We called this style of combination “lacing” because we connect the data slots across in parallel fashion. The vertical arrows denote this mapping behaviour.
[ ]1 ⮆ [ ]n ⟹ [ ]n and also [ ]n ⮆ [ ]1 ⟹ [ ]n
We are also familiar with the behaviour of combining lists with single values producing lists with the same size as the input list. We used this behaviour extensively in order to transform sequences.
[ ]n ⮆ [ ]m ⟹ [ ]max( n, m )
We did not use combinations of lists with different sizes; it is a bit sacrilegious. The result is also a list with as many items as the maximum number of items of the inputs. The diagram below shows an example of 2 with 3. The key take away here is the use of the so called maximal matching size rule.
To deal with the fact that we do not have enough values in the smaller list, we use duplication to make both lists equal in size and then perform the “lacing” technique seen above. The diagram below shows the duplicated values.
If you now revisit the list with a single value scenario, you can see that it is practically falling under this paradigm: we maximally match sizes and perform lacing.
[ ]n ⤨ [[ ]]m ⇛ [[ ]n]m but [[ ]]n ⤨ [ ]m ⇛ [[ ]m]n
The Cartesian product is not symmetric! You can see the result of combining 3 with 2 and 2 with 3 item lists below. This works in our favour because we can construct either output shapes i.e. 3 x 2 or 2 x 3 tables, by switching the wires / operands order.
Combining the values of a 1D list and 2D table (or list of lists) is not straight forward. They have different shapes, so we need to match their dimensions. Again we use the concept of maximal matching and promote the 1D list to a 2D table by just placing it inside another list ie. [ ] -⤍ [[ ]] by grafting. In this way we have matched the nesting depth of both operands. This is also known as the rank matching shape rule. Finally, we perform lacing element by element. Let’s look at a step-by-step example:
To conclude this detour through list shape and size algebras we just need to consider that all these were performed such that we can consistently combine values. In fact, this generalizes to higher dimensions of data containers but here we will stick to 1D lists and 2D tables. There is no need to memorize the rules but in the case the results of some operation are not obvious, you can come back and retrace the steps above.
The following video demonstrates the concept of a Cartesian products i.e. computing the pair-wise combinations of all columns and rows to build a grid []. We start by using basic arithmetic operations, then we move to an example using numbers and letters, along the spreadsheet analogy, and finally we build a grid of points.
