Add Fields
Adds one or more fields to the input documents. Returns all existing fields plus the newly added field(s).
If an added field has the same name as an existing field, that existing field is overwritten. This allows the modification (including nullification) of existing field(s). Below is an explanation on keeping the existing field meta if desired.
A similar stage is Select Fields. It too has the ability to add fields, but it returns only selected existing fields. See also a related stage Rename Fields.
Stage as JSON
For convenience there are two ways to declare it.
As an object (with keys and values)
// As an object (with keys, values)
{
<field1_name>: <expression>,
<field2_name>: <expression>,
...
// you may also:
<create-fields-statement>
}
// Example
{
"qf24a_08r": "(waveid==2 and qf17==2) ? 0 : qf24a_08",
"yyyyq": "getYear(timestamp) * 10 + getQuarter(timestamp)"
}
As an array (with name and syntax)
// As an array
[
{ name: "<field1_name>", syntax: "<expression>" },
{ name: "<field2_name>", syntax: "<expression>" },
...
]
// Example
[
{ name: "qf24a_08r", syntax: "(waveid==2 and qf17==2) ? 0 : qf24a_08" },
{ name: "yyyyq", syntax: "getYear(timestamp) * 10 + getQuarter(timestamp)" }
]
Meta
When adding a new field, it gets a name. But you may wish to additionally give it a
label and valueLabels (collectively referred to as field meta). You may
pull field meta from an existing field or create it from scratch.
When renaming a field
If you are simply renaming a field, you'll probably wish to pull field meta from the original field.
// using object addFields layout:
{
$addFields: {
"awareness": "Q3", // automatically pulls field meta from Q3
}
}
// using array addFields layout:
{
$addFields: [
// the use of "field" means that meta (field and valueLabels) will be loaded from the provided field
{ name: "awareness", field: "Q3" }, // BE SURE TO USE "field" rather than "syntax"
// THIS WILL NOT take meta from existing field:
{ name: "awareness", syntax: "Q3" }, // any time syntax is explicitly provided, meta is blank
// i might change this since if the syntax is a field name only (to match functionality of object layout)
]
}
When recoding a field
Since recoding a field requires the creation of a new field (even if it has the same name as before), you must declare the field meta (label and valueLabels). There are a few ways to do it:
{
$addFields: [
// in this example q1R gets no meta (no label, no valueLabels)
{
name: "q1R",
syntax: "isnull(q1,0)"
},
// in this example q1R gets label from q1 (but no valueLabels)
{
name: "q1R",
syntax: "isnull(q1,0)"
label: { $fromField: "q1" },
}
// in this example q1R gets valuelabels from q1
{
name: "q1R",
syntax: "isnull(q1,0)",
label: "Awareness re-based to all respondents",
valueLabels: { $fromField: "q1" } // take valueLabels from q1
}
// in this example q1R gets all meta (label and valueLabels) from q1
{
name: "q1R",
syntax: "isnull(q1,0)",
metaFrom: "q1" // pulls label and valueLabels from q1
}
]
}
Build field meta from scratch
{
$addFields: [
{
name: "q7_rollup",
syntax: "q7 in (3,4,5) ? 3 : q7",
label: "My great rollup of Q7",
valueLabels: [
{ val: 1, label: "Item 1" },
{ val: 2, label: "Item 2" },
{ val: 3, label: "Item 3, 4 or 5" },
]
}
]
}
Large bank of recodes
{
$addFields: [
// recode: convert missing values to zero in Q1_* series
{ name: "Q1_1r", syntax: "isnull(Q1_1,0)", metaFrom: "Q1_1" },
{ name: "Q1_2r", syntax: "isnull(Q1_2,0)", metaFrom: "Q1_2" },
{ name: "Q1_3r", syntax: "isnull(Q1_3,0)", metaFrom: "Q1_3" },
{ name: "Q1_4r", syntax: "isnull(Q1_4,0)", metaFrom: "Q1_4" },
{ name: "Q1_5r", syntax: "isnull(Q1_5,0)", metaFrom: "Q1_5" },
{ name: "Q1_6r", syntax: "isnull(Q1_6,0)", metaFrom: "Q1_6" },
{ name: "Q1_7r", syntax: "isnull(Q1_7,0)", metaFrom: "Q1_7" },
{ name: "Q1_8r", syntax: "isnull(Q1_8,0)", metaFrom: "Q1_8" },
{ name: "Q1_9r", syntax: "isnull(Q1_9,0)", metaFrom: "Q1_9" },
{ name: "Q1_10r", syntax: "isnull(Q1_10,0)", metaFrom: "Q1_10" },
]
}
Unrolling a set of pick vars (NOTES ONLY)
todo: flesh this out
{
// the hard way
$addFields: [
{ name: "var_oe_100", syntax: "pick1==100 or pick2==100 or pick3==100 or pick4==100 or pick5==100" },
{ name: "var_oe_101", syntax: "pick1==101 or pick2==101 or pick3==101 or pick4==101 or pick5==101" },
{ name: "var_oe_102", syntax: "pick1==102 or pick2==102 or pick3==102 or pick4==102 or pick5==102" },
{ name: "var_oe_103", syntax: "pick1==103 or pick2==103 or pick3==103 or pick4==103 or pick5==103" },
{ name: "var_oe_104", syntax: "pick1==104 or pick2==104 or pick3==104 or pick4==104 or pick5==104" },
{ name: "var_oe_105", syntax: "pick1==105 or pick2==105 or pick3==105 or pick4==105 or pick5==105" },
{ name: "var_oe_106", syntax: "pick1==106 or pick2==106 or pick3==106 or pick4==106 or pick5==106" },
{ name: "var_oe_107", syntax: "pick1==107 or pick2==107 or pick3==107 or pick4==107 or pick5==107" },
]
//{ name: "var_oe_{x}", syntax: "pick1=={x} or pick2=={x} or pick3=={x} or pick4=={x} or pick5=={x}" },
// the easy way
$addFields: [
{
$unroll: {
vars: ["pick1", "pick2", "pick3", "pick4", "pick5"],
nameTemplate: "var_oe_{value}",
label: something([
{ val: 100, label: "todo" },
{ val: 101, label: "todo" },
{ val: 102, label: "todo" },
{ val: 103, label: "todo" },
{ val: 104, label: "todo" },
{ val: 105, label: "todo" },
{ val: 106, label: "todo" },
{ val: 107, label: "todo" },
])
}
}
]
}