You can simply take the last value with array indexing. In your example: l(a)[end]

yes, I figured. But would be nice to have functionality built-in, since ideally one would like to feed the last value as input to let's say a Dense layer, but if the LSTM only returns a sequence then I can't make sense how a Dense layer is supposed to process it. I was looking into the source code, it doesn't seem to provide any kwargs such that one could set return_sequences=false. Unless I am obviously missing something there, I would probably implement this functionality myself.

I didn't catch it at first, but in your example, what you actually have is a Batch. The return value is equal to a single timestep of a LSTM

If in that case, if we reshape to (features,timesteps,batches) and we feed it as one batch, it gives nonsensical results, as can be seen here:

Another minimal example shows that if we feed a batch of 1 sample, it still returns a sequence equal to the length of timesteps.

I believe, as an easy fix, one could also plug (x->x[:,end]) in the Chain so that the succeeding layers are fed only the last state.
Chain(LSTM(n,m), x -> x[:,end], Dense(m,p))

Each time you call the LSTM is a timestep. You can use broadcasting, but not in the way you used.
You'd need to broadcast the LSTM forward pass over an array distributed in time, i.e. each array is a timestep.
So a = [randn(2,10) for _ in 1:20] would be an input, with batchsize 10 and 20 timesteps.
Then you'd call l.(a), broadcasting the LSTM forward pass over the array in time.
To keep the last element and apply a Dense layer to it, it would be simply d(l.(a)[end])

Another, very Julian, possibility would be mapslice, with time as the last dimension (good for memory management, IIRC), but I'm not sure how well is mapslice works with Zygore right now. @DhairyaLGandhi and @CarloLucibello could comment on that I think.

Seems quite confusing and unconventional. If I have 1 timeseries of 2 categorical values like rand([1,-1],100) and I make it onehot and reshape it to (2,1,100) then what will the input dimensions of an LSTM layer? If I use LSTM(2,1) then it gives nonsensical output like here:

You shouldn't reshape it if you want to broadcast. You should keep each timestep as an array (Input size X Batch size), and then broadcast over that.