RFC: Primary Dimension Field Expansion

[cdeszaq]

RFC: Primary Dimension Field Expansion

• Feature Name: Primary Dimension Field Expansion
• Start Date: 2016-11-11
• RFC PR:
• Fili Issue:

Summary

Fili Dimensions having only 1 "core" field, the key field, has gotten us a long way, but it has become obvious that all Dimensions have more core fields than that, which serve other common use-cases. We propose changing the name of the key field to better align to general database terminology, adding 2 additional core fields (display, and reference), and to surface these three core fields in the Dimension resource so that clients can make use of them.

Motivation

As Fili usage has expanded, additional use-cases have surfaced around Dimensions, particularly when it comes to core fields of a Dimension. For example, programmatic clients, like web UIs, often want more information about the fields of a Dimension in order to provide a good user experience. This includes things like "Which fields are unique?", "Which fields are the default search fields?", "Which fields are alternate identifiers?", and "Which fields should I use when displaying a DimensionRow in various contexts?" Some of these use cases are already handled, or are mostly handled. For example, Dimension already has the concepts of a "key" field and of "default dimension fields", but we don't fully solve for those use cases because those concepts are not surfaced in the Dimension resource.

To handle more use cases, we'd like to do 2 main things, and one small change to bring what already exists in line with this proposal:

1. Add the concept of a "display" field

   This core field indicates to a client which field should be used to display or label a particular DimensionRow. An example of this would be to label the series on a chart, or the field to put in a table when grouping by a Dimension if a user didn't choose any other field to show. This would generally align to how an end-user would talk about a particular DimensionRow, and in many cases it would be something like a "name" field.

2. Add the concept of a "reference" field

   The "reference" field can be thought of as an "alternate identifier", and would typically be how an end-user might identify a DimensionRow, even though it may not be a truly unique value. The reference field should generally be "mostly unique", in that it will identify things well enough for an end-user to be able to use, but it isn't unique enough for Fili to use as it's "key" field.

   A common example of this would be the difference between a truly unique surrogate key field that holds UUIDs, and a mostly-unique "id" field that holds a more human-friendly auto-incrementing number. If the data held combined records from two different systems that both ad an "id" field, that field wouldn't be unique since the ids might overlap, but the "key" field is guaranteed to be unique. In that example, and end-user would prefer to use the id, since it's shorter, more friendly, and they may never want to see or use the surrogate key.

3. Rename the "key" field concept to "primary"

   While "key" has served Fili well to denote the primary key field of the Dimension, key generally means any unique field, and doesn't really indicate the primary or principal nature of the field in the way that Fili interprets it. So, to better align the nomenclature to both the usage and semantics that Fili puts on the key field, as well as to how other more widely-used relational database systems refer to what Fili calls the key field, we propose to change the name of the concept from "key" field to "primary" field.

As part of these conceptual changes to Dimension, we'd also like to surface all 3 of these concepts through the Dimension resource so that API clients can utilize the information. In particular, the expectation is that this information can help drive programmatic clients like UI applications. This information may also be useful for API "power users", by giving them some insight into additional semantics of DimensionFields, and could allow for richer experiences like default filtering, where the field isn't specified, on columns other than the "key" column.

Detailed design

This is the bulk of the RFC. Explain the design in enough detail for somebody familiar
with the language to understand, and for somebody familiar with the compiler to implement.
This should get into specifics and corner-cases, and include examples of how the feature is used.

TBD

Drawbacks

We can see 4 main drawbacks to this change:

1. More concepts == more complexity

• May be too restrictive
  Mitigation: Can make new fields optional
  Mitigation: It's OK for 1 field to be in multiple "roles"
  ie. could have a 1-field dimension that has it's single field as all 3 of them.

• More to config
  Mitigation: Can allow to be null
  Mitigation: Can default to... something?
  Mitigation: Should be not be a breaking config change

• More complexity
  Mitigation: The complexity is likely already there, Fili just doesn't capture it or expose it
  Mitigation: Can solve through documentation
  Mitigation: Can hide the complexity via default fields or config defaults, so only those who need it or go looking for it have to deal with the complexity

2. Venturing closer to "display"
   Mitigation: Not really. Just describing semantics, and not dictating how a client leverages that information
   Mitigation: We are already there (even closer) with some of the other fields like "description" and "longName"

3. Changing existing concept's name may break those relying on it
   Mitigation: Can deprecate existing interface and make existing interface just proxy to new interface

4. Api change
   Mitigation: Adding, not changing or removing, so is backwards compatible
   Mitigation: could rev the API (though we don't have a great way to do this)

Alternatives

Alt design:

1. "tagging" fields on the existing object

What if not?

1. Clients need to hard-code the info, or get it from some other source

Unresolved Questions

• How to configure
• What to use as default value

[cdeszaq]

Haven't gotten time to update this yet, so I'll either re-open once I've updated, or open a new one.