-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add integer, float, string, date, and datetime conversion functions #1437
Conversation
refactor computed fn Use shared pointers for t_computed_expression add date() and datetime(), remove m_none, clean up computed fns datetime() takes POSIX timestamp
string() parse strings in integer() and float()
ec35801
to
f705593
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! Thanks for the PR!
@@ -230,7 +256,7 @@ t_computed_expression_parser::init() { | |||
.disable_base_function(exprtk::parser<t_tscalar>::settings_store::e_bf_max); | |||
} | |||
|
|||
t_computed_expression | |||
std::shared_ptr<t_computed_expression> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are these all shared_ptr
now? Just curious
|
||
#ifdef PSP_ENABLE_WASM | ||
// check for overflow | ||
if (number > std::numeric_limits<std::int32_t>::max() || number < std::numeric_limits<std::int32_t>::min()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I appreciate the attention to detail here, but it's worth noting we definitely don't handle this correctly at the JS boundaries and have frequent overflows .. though perhaps this doesn't need to be said #1346
This PR adds a set of type conversion functions to the expression language, which will allow users to quickly clean/coerce their data within Perspective if it is inferred as the wrong format:
integer(x)
: converts a column or scalar of any type to an integer. InPerspective.js
, integers are 32-bit, thus values that overflow/underflow will be output asnull
.float(x)
: converts a column or scalar of any type to a float.string(x)
: converts a column or scalar of any type into a string.date
values are output using the formatYYYY-MM-DD
, anddatetime
values are output using the formatYYYY-MM-DD HH:MM:SS
. Conversion of floats to strings can lose precision.date(year, month, day)
: given a year, month (1-12), and day, create a newdate
value. This allows for parsing of numerical date formats that were previously impossible to cast to a date, such asYYYYMMDD
. Parsing can be done with a simple expression:datetime(timestamp)
: given a POSIX timestamp of milliseconds since epoch, create a new datetime value. This allows for timestamp columns which would usually be inferred asfloat
(unless the table is provided with a schema marking the column as adatetime
) to be converted into datetimes. Because of the addition of scalars, converting from a seconds-since-epoch timestamp to a milliseconds timestamp is simple:Additionally, this PR removes a little debug cruft left over from #1431, and edits the Monaco autocomplete definitions to include the proper parameters for
bucket
,if/else
, andvar
.