@imatiach-msft Sorry for the inconvenience, but can you please rebase to the latest master to fix CI checks?

@StrikerRUS done!

Hey @imatiach-msft,

Can you please resolve conflicts that appeared after merging #2850?

@StrikerRUS done! Resolved conflicts, updated PR.

ping @guolinke @StrikerRUS thanks!

@imatiach-msft
The diff looks nice to me, but unfortunately I have no idea what is going on here, sorry! 😃
static TYPE *new_##NAME
%{}
...

@StrikerRUS this is just a slightly modified version of the SWIG file here which we were already using in the previous %array_functions call:
https://github.com/swig/swig/blob/master/Lib/carrays.i
note how the values with "int" type are now just replaced to have "int64_t" type.
Hope this helps clear up any confusion!

Hello @imatiach-msft :),

If we're going to change that, then can we use an unsigned type like size_t? It's also the underlying type being used in the operator new allocation and negative values shouldn't be allowed.

Also,

SWIG's carrays.i, is NOT memory-allocation safe, which means your whole application can crash just because the OS kills your underlying C process.
Before talking with you, I posted an issue at SWIG's repo to see if they wanted to fix it but they never answered back: swig/swig#1755.

So my suggestion is since we're changing the Java API anyways, just fix it ourselves. We could throw an exception or even easier, use new (nothrow). Anyone who wishes to handle that case just has to test in Java for the null pointer, all other users who are not concerned with such issues (non-production) can keep working like before.

What do you say?

@AlbertoEAF "then can we use an unsigned type like size_t"
I think long is way more than enough in terms of size for the use-case. I'm not sure how I would call this from the java/scala code whereas I've already tested this with int64_t.

@AlbertoEAF "So my suggestion is since we're changing the Java API anyways, just fix it ourselves"

Agree that it should be fixed, but can this be part of a separate PR? This issue is much more pressing and I really want to fix it so I can publish a new lightgbm jar for customers with very large data. The additional change would require more integration testing on my side.

gently ping @guolinke for the review