Add Fields
Adds one or more fields to the input documents. Returns all existing fields plus the newly added field(s).
If an added field has the same name as an existing field, that existing field is overwritten. This allows the modification (including nullification) of existing field(s). Below is an explanation on keeping the existing field meta if desired.
A similar stage is Select Fields. It too has the ability to add fields, but it returns only selected existing fields. See also a related stage Rename Fields.
Stage as JSON
For convenience there are two ways to declare it.
As an object (with keys and values)
// As an object (with keys, values)
{
<field1_name>: <expression>,
<field2_name>: <expression>,
...
// you may also:
<create-fields-statement>
}
// Example
{
"qf24a_08r": "(waveid==2 and qf17==2) ? 0 : qf24a_08",
"yyyyq": "getYear(timestamp) * 10 + getQuarter(timestamp)"
}
As an array (with name and syntax)
// As an array
[
{ name: "<field1_name>", syntax: "<expression>" },
{ name: "<field2_name>", syntax: "<expression>" },
...
]
// Example
[
{ name: "qf24a_08r", syntax: "(waveid==2 and qf17==2) ? 0 : qf24a_08" },
{ name: "yyyyq", syntax: "getYear(timestamp) * 10 + getQuarter(timestamp)" }
]
Meta
When adding a new field, it gets a name. But you may wish to additionally give it a
label and valueLabels (collectively referred to as field meta). You may
pull field meta from an existing field or create it from scratch.
When renaming a field
If you are simply renaming a field, you'll probably wish to pull field meta from the original field.
// using object addFields layout:
{
"awareness": "Q3", // automatically pulls field meta from Q3
}
// using array addFields layout:
[
{ name: "awareness", syntax: "Q3" }, // automatically pulls field meta from Q3
]
When recoding a field
[
{ "name": "Q1R", "syntax": "Q1 ?? 0" }, // automatically pulls field meta from Q1
{ "name": "q7_rollup", "syntax": "q7 in (3,4,5) ? 3 : q7" } // parser not smart enough to glean meta
]
Build field meta from scratch
[
{
"name": "q7_rollup",
"syntax": "q7 in (3,4,5) ? 3 : q7",
"label": "My great rollup of Q7",
"valueLabels": [
{ "value": 1, "label": "Item 1" },
{ "value": 2, "label": "Item 2" },
{ "value": 3, "label": "Item 3, 4 or 5" }
]
}
]
Large bank of recodes
! i probably won't do this
[
// recode: convert missing values to zero in Q1_* series
{ name: "Q1_1r", syntax: "isnull(Q1_1,0)", metaFrom: "Q1_1" },
{ name: "Q1_2r", syntax: "isnull(Q1_2,0)", metaFrom: "Q1_2" },
{ name: "Q1_3r", syntax: "isnull(Q1_3,0)", metaFrom: "Q1_3" },
{ name: "Q1_4r", syntax: "isnull(Q1_4,0)", metaFrom: "Q1_4" },
{ name: "Q1_5r", syntax: "isnull(Q1_5,0)", metaFrom: "Q1_5" },
{ name: "Q1_6r", syntax: "isnull(Q1_6,0)", metaFrom: "Q1_6" },
{ name: "Q1_7r", syntax: "isnull(Q1_7,0)", metaFrom: "Q1_7" },
{ name: "Q1_8r", syntax: "isnull(Q1_8,0)", metaFrom: "Q1_8" },
{ name: "Q1_9r", syntax: "isnull(Q1_9,0)", metaFrom: "Q1_9" },
{ name: "Q1_10r", syntax: "isnull(Q1_10,0)", metaFrom: "Q1_10" },
]
Unrolling a set of pick vars (NOTES ONLY)
note
todo: flesh this out
As syntax:
[
{ "name": "x_item100", "syntax": "pick1==100 ? x_pick1 : pick2==100 ? x_pick2 : pick3==100 ? x_pick3 : null" },
{ "name": "x_item101", "syntax": "pick1==101 ? x_pick1 : pick2==101 ? x_pick2 : pick3==101 ? x_pick3 : null" },
{ "name": "x_item102", "syntax": "pick1==102 ? x_pick1 : pick2==102 ? x_pick2 : pick3==102 ? x_pick3 : null" },
{ "name": "x_item103", "syntax": "pick1==103 ? x_pick1 : pick2==103 ? x_pick2 : pick3==103 ? x_pick3 : null" },
{ "name": "x_item104", "syntax": "pick1==104 ? x_pick1 : pick2==104 ? x_pick2 : pick3==104 ? x_pick3 : null" },
{ "name": "x_item105", "syntax": "pick1==105 ? x_pick1 : pick2==105 ? x_pick2 : pick3==105 ? x_pick3 : null" },
{ "name": "x_item106", "syntax": "pick1==106 ? x_pick1 : pick2==106 ? x_pick2 : pick3==106 ? x_pick3 : null" },
{ "name": "x_item107", "syntax": "pick1==107 ? x_pick1 : pick2==107 ? x_pick2 : pick3==107 ? x_pick3 : null" },
]
As javascript:
() => {
return [100, 101, 102, 103, 104, 105, 106, 107].map(val => ({
name: `x_item${val}`,
syntax: `pick1==${val} ? x_pick1 : pick2==${val} ? x_pick2 : pick3==${val} ? x_pick3 : null`,
})
}
As javascript with field meta (label, valueLabels):
() => {
const entries = [
{ val: 100, label: "Item 100" },
{ val: 101, label: "Item 101" },
{ val: 102, label: "Item 102" },
{ val: 103, label: "Item 103" },
{ val: 104, label: "Item 104" },
{ val: 105, label: "Item 105" },
{ val: 106, label: "Item 106" },
{ val: 107, label: "Item 107" }
];
return entries.map(entry => {
return {
name: `x_item${entry.val}`,
syntax: `pick1==${entry.val} ? x_pick1 : pick2==${entry.val} ? x_pick2 : pick3==${entry.val} ? x_pick3 : null`,
label: entry.label,
valueLabels: [
{ value: 1, label: "Yes" },
{ value: 2, label: "No" },
{ value: 3, label: "Don't Know" },
]
}
})
}
// something like this???
(context) => {
const picks = [...]
const entries = [...]
return unroll_pick(entries, picks, (entry, pick) => `todo`)
}