Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

window function normalization #6

Closed
renerichter opened this issue Jul 21, 2021 · 3 comments
Closed

window function normalization #6

renerichter opened this issue Jul 21, 2021 · 3 comments

Comments

@renerichter
Copy link
Contributor

renerichter commented Jul 21, 2021 •
edited
Loading

Hiho,
thank you for this awesome project. I played a bit around with it and tried all tiling and windowing options, but it seems to me that in the stitched image the windowing functions do not sum up to 1 (=100%), but to 0. I tested it with the following code:

import numpy as np
from tiler import Tiler, Merger
import matplotlib.pyplot as plt
imsize = 64
im = np.reshape(np.arange(imsize*imsize),[imsize,imsize])
batch_size = 10
overlap = 5
tiler_mode = 'wrap'
windows_supported = ['boxcar', 'triang', 'blackman', 'hamming', 'hann', 'bartlett', 'flattop', 'parzen', 'bohman', 'blackmanharris', 'nuttall', 'barthann', 'overlap-tile']
tiler = Tiler(data_shape=im.shape,
                tile_shape=(16,16),
                channel_dimension=None,overlap=overlap,mode=tiler_mode,constant_value=1.0)
#Tiler.TILING_MODES == ['constant', 'drop', 'irregular', 'reflect', 'edge', 'wrap']
tiles_in_batches = [batch for _, batch in tiler(im, batch_size=batch_size)]
merged_images = []

for mwin in windows_supported:
    merger = Merger(tiler,window=mwin)
    #Merger.SUPPORTED_WINDOWS == ['boxcar', 'triang', 'blackman', 'hamming', 'hann', 'bartlett', 'flattop', 'parzen', 'bohman', 'blackmanharris', 'nuttall', 'barthann', 'overlap-tile']
    
    for batch_id, batch in enumerate(tiles_in_batches):
        merger.add_batch(batch_id, batch_size, batch)

    imf = merger.merge(unpad=True,argmax=False)
    merged_images.append(imf)
merged_images = np.array(merged_images)
fig,ax = plt.subplots(nrows=4,ncols=4,figsize=[14,14])
axm = ax.flatten()
for m,mim in enumerate(merged_images): 
    ima = axm[m].imshow(mim)
axm[m+1].imshow(im)
plt.show()

which results in

tiling_test

where the last image is the original (to-be-tiled and merged) image and the others are the different merges from all possible merging options.

What am I doing wrong here?
@the-lay
Copy link
Owner

the-lay commented Jul 21, 2021

Hi @renerichter ! Great to see the interest!

Unfortunately I still haven't implemented automatic calculation of window sizes and overlap to get constant 1.0 "weight" in Merger. It is in the roadmap and I have to admit it has been there for too long! For now, you have to calculate "correct" sizes yourself and/or apply padding like in the examples/2d_overlap_tile.py.

You could divide the merged data by the number of times each element has been seen in tiles. This would work properly only for boxcar (constant) window: imf = merger.merge(unpad=True, argmax=False) / merger.data_visits[:imsize, :imsize]
I think I will add such normalization as another flag for Merger.merge() next time I work on the project.

normalize_tiler

Hope that helps!

@renerichter
Copy link
Contributor Author

renerichter commented Sep 27, 2021 •
edited
Loading

I added #7 and would like to discuss the merging strategy.
Building onto my pull-request the following test-code is applied

Test Code

import numpy as np
from tiler import Tiler,Merger
import matplotlib.pyplot as plt

# %% Test example for online

# parameter
data_shape = [1,256,256]
tile_shape = np.array([1,64,64])
overlap=np.array([0,7,20])
tiler_mode = 'wrap'
windows_supported = ['boxcar', 'triang', 'blackman', 'hamming', 'hann', 'bartlett', 'flattop', 'parzen', 'bohman', 'blackmanharris', 'nuttall', 'barthann', 'overlap-tile']

# image
xy = np.ogrid[0:data_shape[-2],0:data_shape[-1]]
xy = np.sqrt(xy[0]*np.transpose(xy[0])+xy[1]*np.transpose(xy[1]))
im = np.cos(4*xy*np.pi/np.max(xy))[np.newaxis]
merged_images = [im,]

# tile me
tiler = Tiler(data_shape=data_shape, tile_shape=tile_shape,overlap=tuple(overlap),get_padding=True)
im_padded = tiler.pad_outer(im,tiler.pads)
weights_sums = [np.ones(im_padded.shape),]

for mwin in windows_supported:
    merger = Merger(tiler,window=mwin)
    #Merger.SUPPORTED_WINDOWS == ['boxcar', 'triang', 'blackman', 'hamming', 'hann', 'bartlett', 'flattop', 'parzen', 'bohman', 'blackmanharris', 'nuttall', 'barthann', 'overlap-tile']
    
    for tile_id, tile in tiler(im_padded):
        processed_tile = tile # lambda x: x
        merger.add(tile_id, processed_tile)

    imf = merger.merge(data_orig_shape=data_shape)
    merged_images.append(imf)
    weights_sums.append(merger.weights_sum)

# plot images
merged_images = np.array(merged_images)[:,0]
fig,ax = plt.subplots(nrows=4,ncols=4,figsize=[14,14])
plt_titles = ['reference',]+windows_supported
axm = ax.flatten()
for m,mim in enumerate(merged_images): 
    ima = axm[m].imshow(mim,interpolation='None')
    axm[m].set_title(plt_titles[m])
plt.suptitle('Resulting Images')
plt.tight_layout()
plt.show() 

# plot images
weights_sums = np.array(weights_sums)[:,0]
fig2,ax2 = plt.subplots(nrows=4,ncols=4,figsize=[14,14])
plt_titles = ['reference',]+windows_supported
axm = ax2.flatten()
inv_weights = merger.norm_by_weights((weights_sums[0])[np.newaxis], weights=weights_sums)
inv_weights/= np.max(inv_weights,axis=(-2,-1),keepdims=True)
for m,invm in enumerate(inv_weights): 
    ima = axm[m].imshow(invm**0.002,interpolation='None')
    axm[m].set_title(plt_titles[m])
plt.suptitle('Inverse Weights-maps **0.002 (for display)')
plt.tight_layout()
plt.show() 

# differences
for m,mim in enumerate(merged_images):
    print(f"All close for window_func={plt_titles[m]}?\t {np.allclose(mim,im[0])}")

The results are:

Resulting Images:

2021-09-27-09-04-21-tiler_test_res

Weights used for reweighting:

2021-09-27-09-29-05-inverse_wegihts_map

Comparison between Reference and re-merged images

All close for window_func=reference?     True
All close for window_func=boxcar?        True
All close for window_func=triang?        TrueM
All close for window_func=blackman?      True
All close for window_func=hamming?       True
All close for window_func=hann?  True
All close for window_func=bartlett?      True
All close for window_func=flattop?       False
All close for window_func=parzen?        True
All close for window_func=bohman?        True
All close for window_func=blackmanharris?        True
All close for window_func=nuttall?       True
All close for window_func=barthann?      True
All close for window_func=overlap-tile?  False

For the reweighting I interpreted the calculation steps to be like if one would calculate a center-of-mass kind of thing. Hence,

  • multiplying each tile with the window
  • summing the tiles into the final image
  • dividing by the summed weights window-function

could reproduce the wanted normalization of the output. On the other hand, if the window-functions are normed and symmetric than the final division by the weights would just undo the prior weighting of the tiles. Hence I believe I have a thinking error somehow here. What do you think?

PS: Flat-top normalization needs to be fixed...

@the-lay the-lay mentioned this issue Oct 10, 2021
Merged
@the-lay
Copy link
Owner

the-lay commented Dec 14, 2021

Merged #7.
I will close this issue, but please feel free to open new issues/discussions!

@the-lay the-lay closed this as completed Dec 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@the-lay @renerichter