addEventLines & yearmon index #353

paessens · 2021-05-15T17:44:21Z

Description

I could not use addEventLines having both xts' indexed with yearmon format

Expected behavior

same as with YYYY-MM-DD indexes

Minimal, reproducible example

require(magrittr)
require(xts)
require(ecb)

data.frame(
  begin = c("1998-10-01","2000-04-01","2001-10-01","2008-10-01","2009-10-01","2020-04-01"),
  events = c("LTCM","dot com bubble","9/11","Lehman Bros.","Eurozone Crisis","Pandemic")) %>%
  read.zoo(FUN=as.yearmon) %>%
  as.xts() -> historical

'ICP.M.U2.N.000000.4.ANR' %>% ecb::get_data() %>%
  .[c('obstime','obsvalue')] %>%
  read.zoo(FUN=as.yearmon) %>% as.xts() -> HICP

plot(HICP,
     col='navy',lty=1,lwd=1.5, cex=0.5,
     grid.ticks.lty="dotted")
addEventLines(historical, col="#3357FF", cex=1,lty=1.5,lwd=2, srt=90, pos=2,offset=-.6,on=0)

Session Info

R version 4.0.5 (2021-03-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 10 (buster)

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.5.so

locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8
[6] LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices datasets utils methods base

other attached packages:
[1] magrittr_2.0.1 ecb_0.4.0 xts_0.12.1 zoo_1.8-9

loaded via a namespace (and not attached):
[1] Rcpp_1.0.6 lattice_0.20-44 XML_3.99-0.6 rsvg_2.1.2 digest_0.6.27 plyr_1.8.6 R6_2.5.0 grid_4.0.5
[9] rsdmx_0.6 evaluate_0.14 httr_1.4.2 quantmod_0.4.18 rlang_0.4.11 cachem_1.0.4 renv_0.13.2 curl_4.3.1
[17] TTR_0.24.2 rmarkdown_2.8 tools_4.0.5 xfun_0.22 yaml_2.2.1 fastmap_1.1.0 compiler_4.0.5 htmltools_0.5.1.1
[25] memoise_2.0.0 knitr_1.33

paessens · 2021-05-15T17:47:29Z

error message:

Error in index<-.xts(*tmp*, value = order.by) :
unsupported ‘index’ index type of class ‘integer’
In addition: Warning messages:
1: In as_numeric(YYYY) : NAs introduced by coercion
2: In as_numeric(YYYY) : NAs introduced by coercion
3: In as_numeric(YYYY) : NAs introduced by coercion
4: In .parseISO8601(ii, .index(x)[1], .index(x)[nr], tz = tz) :
cannot determine first and last time from Jan 1997/Apr 2021

paessens · 2021-05-15T18:37:58Z

this works as expected (converting monthy data to daily, wrt date format)

require(magrittr)
require(xts)
require(ecb)

data.frame(
begin = c("1998-10-01","2000-04-01","2001-10-01","2008-10-01","2009-10-01","2020-04-01"),
events = c("LTCM","bubble.com ","9/11","Lehman Bros.","Eurozone Crisis","Pandemic")) %>%
read.zoo() %>%
as.xts() -> historical

'ICP.M.U2.N.000000.4.ANR' %>% ecb::get_data() %>%
.[c('obstime','obsvalue')] -> HICP

HICP$obstime %<>% paste0('-01')
HICP$obstime %<>% as.Date()
HICP %<>% read.zoo() %>% as.xts()

plot(HICP, col='navy',lty=1,lwd=1.5,cex=0.5,grid.ticks.lty="dotted")
addEventLines(historical, col="#FF480B", cex=1,lty=1.5,lwd=2, srt=90, pos=2,offset=-.6,on=0)

joshuaulrich · 2021-06-13T16:13:12Z

I can replicate. Here's minimal example that doesn't require installing several other packages:

library(xts)

historical <-
  data.frame(
  begin = c("1998-10-01","2000-04-01","2001-10-01","2008-10-01","2009-10-01","2020-04-01"),
  events = c("LTCM","dot com bubble","9/11","Lehman Bros.","Eurozone Crisis","Pandemic"))
historical <- as.xts(read.zoo(historical, FUN=as.yearmon))

HICP <-
structure(c(2, 1.8, 1.6, 1.3, 1.4, 1.4, 1.5, 1.7, 1.6, 1.5, 1.6, 1.5, 1.1, 1.1, 1.1, 1.4, 1.3, 1.3, 1.3,
    1.1, 1, 0.9, 0.8, 0.8, 0.8, 0.7, 0.9, 1.1, 0.9, 0.9, 1, 1.1, 1.2, 1.3, 1.5, 1.7, 1.9, 1.9, 2, 1.7,
    1.8, 2.1, 2.1, 2, 2.4, 2.4, 2.5, 2.5, 2, 2, 2.2, 2.7, 3.1, 2.9, 2.5, 2.3, 2.2, 2.2, 2, 2.1, 2.6,
    2.5, 2.5, 2.4, 2.1, 1.9, 2, 2.1, 2.1, 2.3, 2.3, 2.3, 2.1, 2.4, 2.4, 2.1, 1.8, 2, 1.9, 2, 2.1, 2,
    2.2, 2, 1.8, 1.6, 1.7, 2.1, 2.5, 2.4, 2.3, 2.4, 2.1, 2.4, 2.2, 2.3, 1.9, 2.1, 2.2, 2.1, 2, 2, 2.1,
    2.2, 2.6, 2.5, 2.3, 2.3, 2.4, 2.4, 2.2, 2.4, 2.5, 2.5, 2.4, 2.3, 1.8, 1.6, 1.8, 1.9, 1.8, 1.8, 1.9,
    1.9, 1.9, 1.9, 1.8, 1.7, 2.1, 2.5, 3.1, 3.1, 3.2, 3.3, 3.6, 3.3, 3.7, 3.9, 4.1, 3.8, 3.6, 3.2, 2.1,
    1.6, 1.1, 1.2, 0.6, 0.6, 0, -0.1, -0.6, -0.2, -0.3, -0.1, 0.5, 0.9, 0.9, 0.8, 1.6, 1.6, 1.7, 1.5,
    1.7, 1.6, 1.9, 1.9, 1.9, 2.2, 2.3, 2.4, 2.7, 2.8, 2.7, 2.7, 2.6, 2.5, 3, 3, 3, 2.8, 2.7, 2.7, 2.7,
    2.6, 2.4, 2.4, 2.4, 2.6, 2.6, 2.5, 2.2, 2.2, 2, 1.9, 1.7, 1.2, 1.4, 1.6, 1.6, 1.3, 1.1, 0.7, 0.9,
    0.8, 0.8, 0.7, 0.5, 0.7, 0.5, 0.5, 0.4, 0.4, 0.3, 0.4, 0.3, -0.2, -0.6, -0.3, -0.1, 0.2, 0.6, 0.5,
    0.5, 0.4, 0.2, 0.4, 0.1, 0.3, 0.3, -0.1, 0, -0.3, -0.1, 0, 0.2, 0.2, 0.4, 0.5, 0.6, 1.1, 1.7, 2,
    1.5, 1.9, 1.4, 1.3, 1.3, 1.5, 1.6, 1.4, 1.5, 1.3, 1.3, 1.1, 1.4, 1.2, 2, 2, 2.2, 2.1, 2.1, 2.3,
    1.9, 1.5, 1.4, 1.5, 1.4, 1.7, 1.2, 1.3, 1, 1, 0.8, 0.7, 1, 1.3, 1.4, 1.2, 0.7, 0.3, 0.1, 0.3, 0.4,
    -0.2, -0.3, -0.3, -0.3, -0.3, 0.9, 0.9, 1.3, 1.6, 2), .Dim = c(293L, 1L), .Dimnames = list( NULL, NULL),
    index = structure(c(852076800, 854755200, 857174400, 859852800, 862444800, 865123200, 867715200,
    870393600, 873072000, 875664000, 878342400, 880934400, 883612800, 886291200, 888710400, 891388800,
    893980800, 896659200, 899251200, 901929600, 904608000, 907200000, 909878400, 912470400, 915148800,
    917827200, 920246400, 922924800, 925516800, 928195200, 930787200, 933465600, 936144000, 938736000,
    941414400, 944006400, 946684800, 949363200, 951868800, 954547200, 957139200, 959817600, 962409600,
    965088000, 967766400, 970358400, 973036800, 975628800, 978307200, 980985600, 983404800, 986083200,
    988675200, 991353600, 993945600, 996624000, 999302400, 1001894400, 1004572800, 1007164800,
    1009843200, 1012521600, 1014940800, 1017619200, 1020211200, 1022889600, 1025481600, 1028160000,
    1030838400, 1033430400, 1036108800, 1038700800, 1041379200, 1044057600, 1046476800, 1049155200,
    1051747200, 1054425600, 1057017600, 1059696000, 1062374400, 1064966400, 1067644800, 1070236800,
    1072915200, 1075593600, 1078099200, 1080777600, 1083369600, 1086048000, 1088640000, 1091318400,
    1093996800, 1096588800, 1099267200, 1101859200, 1104537600, 1107216000, 1109635200, 1112313600,
    1114905600, 1117584000, 1120176000, 1122854400, 1125532800, 1128124800, 1130803200, 1133395200,
    1136073600, 1138752000, 1141171200, 1143849600, 1146441600, 1149120000, 1151712000, 1154390400,
    1157068800, 1159660800, 1162339200, 1164931200, 1167609600, 1170288000, 1172707200, 1175385600,
    1177977600, 1180656000, 1183248000, 1185926400, 1188604800, 1191196800, 1193875200, 1196467200,
    1199145600, 1201824000, 1204329600, 1207008000, 1209600000, 1212278400, 1214870400, 1217548800,
    1220227200, 1222819200, 1225497600, 1228089600, 1230768000, 1233446400, 1235865600, 1238544000,
    1241136000, 1243814400, 1246406400, 1249084800, 1251763200, 1254355200, 1257033600, 1259625600,
    1262304000, 1264982400, 1267401600, 1270080000, 1272672000, 1275350400, 1277942400, 1280620800,
    1283299200, 1285891200, 1288569600, 1291161600, 1293840000, 1296518400, 1298937600, 1301616000,
    1304208000, 1306886400, 1309478400, 1312156800, 1314835200, 1317427200, 1320105600, 1322697600,
    1325376000, 1328054400, 1330560000, 1333238400, 1335830400, 1338508800, 1341100800, 1343779200,
    1346457600, 1349049600, 1351728000, 1354320000, 1356998400, 1359676800, 1362096000, 1364774400,
    1367366400, 1370044800, 1372636800, 1375315200, 1377993600, 1380585600, 1383264000, 1385856000,
    1388534400, 1391212800, 1393632000, 1396310400, 1398902400, 1401580800, 1404172800, 1406851200,
    1409529600, 1412121600, 1414800000, 1417392000, 1420070400, 1422748800, 1425168000, 1427846400,
    1430438400, 1433116800, 1435708800, 1438387200, 1441065600, 1443657600, 1446336000, 1448928000,
    1451606400, 1454284800, 1456790400, 1459468800, 1462060800, 1464739200, 1467331200, 1470009600,
    1472688000, 1475280000, 1477958400, 1480550400, 1483228800, 1485907200, 1488326400, 1491004800,
    1493596800, 1496275200, 1498867200, 1501545600, 1504224000, 1506816000, 1509494400, 1512086400,
    1514764800, 1517443200, 1519862400, 1522540800, 1525132800, 1527811200, 1530403200, 1533081600,
    1535760000, 1538352000, 1541030400, 1543622400, 1546300800, 1548979200, 1551398400, 1554076800,
    1556668800, 1559347200, 1561939200, 1564617600, 1567296000, 1569888000, 1572566400, 1575158400,
    1577836800, 1580515200, 1583020800, 1585699200, 1588291200, 1590969600, 1593561600, 1596240000,
    1598918400, 1601510400, 1604188800, 1606780800, 1609459200, 1612137600, 1614556800, 1617235200,
    1619827200), tzone = "UTC", tclass = "yearmon"), class = c("xts", "zoo"))

plot(HICP, col='navy',lty=1,lwd=1.5,cex=0.5,grid.ticks.lty="dotted")
addEventLines(historical, col="#FF480B", cex=1,lty=1.5,lwd=2, srt=90, pos=2,offset=-.6,on=0)

Looks like the problem is this line in addEventLines():

    # xdata is HICP and xsubset is ""
    # result is "Jan 1997/May 2021"
    subset.range <- paste(start(xdata[xsubset]), end(xdata[xsubset]),sep="/")

Subsetting would fail when the plotted object has a yearmon index, because the start/end index values are converted to character as the 3-letter month abbreviation and the 4-digit year. That is not a valid ISO-8601 range string, as shown in the example below. x <- apply.monthly(as.xts(sample_matrix[,4]), last) index(x) <- as.yearmon(index(x)) paste(start(x), end(x), sep = "/") [1] "Jan 2007/Jun 2007" Explicitly format the start/end index values as ISO-8601 before combining them into the range string. Fixes #353.

paessens · 2021-06-13T16:50:06Z

Much appreciated, Joshua

I certainly bear on xts for plotting and analysing time series, however, the work around is easy. But the code looks more concise without work arounds ;-) Here is a glimpse of the most recent project ... https://ilj.at/we1

joshuaulrich · 2021-06-13T16:56:20Z

You're welcome! I'd appreciate it if you could test the patch on other data, especially other frequencies (e.g. quarters, days, hours).

paessens · 2021-06-13T17:00:45Z

will do next week if you provide the patch

…

On 13. Jun 2021, at 18:56, Joshua Ulrich ***@***.***> wrote: You're welcome! I'd appreciate it if you could test the patch on other data, especially other frequencies (e.g. quarters, days, hours). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#353 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVW6OYT3LLJIDNGZNM7P2DTSTPL7ANCNFSM446DBCOA>.

joshuaulrich · 2021-06-13T17:17:37Z

Patch is on branch 353-addEventLines-yearmon, in the commit referenced in this issue: 81ef49a

paessens · 2021-06-13T17:20:37Z

copy that

…

On 13. Jun 2021, at 19:17, Joshua Ulrich ***@***.***> wrote: Patch is on branch 353-addEventLines-yearmon, in the commit referenced in this issue: 81ef49a <81ef49a> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#353 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVW6OYI3WEWVHDVJNBFUALTSTR3ZANCNFSM446DBCOA>.

joshuaulrich · 2021-06-30T11:28:10Z

@paessens, have you had a chance to test this?

paessens · 2021-06-30T12:12:45Z

Oh oh, sorry, missed that. Will do tomorrow and come back to you, I promise. Heinrich

…

On 30. Jun 2021, at 13:28, Joshua Ulrich ***@***.***> wrote: @paessens <https://github.com/paessens>, have you had a chance to test this? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#353 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVW6O7F2KKWWPJBR7E6KUDTVL5VJANCNFSM446DBCOA>.

paessens · 2021-07-02T22:56:52Z

alright -- tested and the plot is as expected (three cases: simple date, yearmon and date+time)
script and data is attached for replication
tst-xts.R.txt
ecb_2021-07-02.RData.zip
Rplot-date.pdf
Rplot-yearmon.pdf
Rplot-posix.pdf

well done Joshua
many thanks - much appreciated
Heinrich

joshuaulrich · 2021-07-03T08:35:13Z

Thanks for testing! I've merged this and it will be in the next release.

paessens · 2021-07-03T09:23:38Z

awesome

paessens closed this as completed Jul 2, 2021

joshuaulrich added the bug label Oct 6, 2022

joshuaulrich added this to the 0.12.2 milestone Oct 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

addEventLines & yearmon index #353

addEventLines & yearmon index #353

paessens commented May 15, 2021

paessens commented May 15, 2021

paessens commented May 15, 2021

joshuaulrich commented Jun 13, 2021

paessens commented Jun 13, 2021

joshuaulrich commented Jun 13, 2021

paessens commented Jun 13, 2021 via email

joshuaulrich commented Jun 13, 2021

paessens commented Jun 13, 2021 via email

joshuaulrich commented Jun 30, 2021

paessens commented Jun 30, 2021 via email

paessens commented Jul 2, 2021

joshuaulrich commented Jul 3, 2021

paessens commented Jul 3, 2021

addEventLines & yearmon index #353

addEventLines & yearmon index #353

Comments

paessens commented May 15, 2021

Description

Expected behavior

Minimal, reproducible example

Session Info

paessens commented May 15, 2021

paessens commented May 15, 2021

joshuaulrich commented Jun 13, 2021

paessens commented Jun 13, 2021

joshuaulrich commented Jun 13, 2021

paessens commented Jun 13, 2021 via email

joshuaulrich commented Jun 13, 2021

paessens commented Jun 13, 2021 via email

joshuaulrich commented Jun 30, 2021

paessens commented Jun 30, 2021 via email

paessens commented Jul 2, 2021

joshuaulrich commented Jul 3, 2021

paessens commented Jul 3, 2021