gdal_rasterize: crash when rasterizing on existing file with gdal.Rasterize() API without any option (3.10.0dev only) #10767
Closed
Description
Cf #10741 (comment) and below @elpaso @tbonfort
Initial setup to prepare an output file:
$ rm -f out.tif; gdal_rasterize -ts 10 10 -burn 0 -burn 0 -burn 0 -l poly autotest/ogr/data/poly.shp out.tif
Crash reproducer:
$ gdb --args python -c "from osgeo import gdal; src_ds = gdal.OpenEx('autotest/ogr/data/poly.shp'); out_ds = gdal.OpenEx('out.tif', gdal.OF_UPDATE); gdal.Rasterize(out_ds, src_ds);"
# the stack trace isn't fully reliable as this is on a RelWithDebInfo optimized build:
x00007ffff5f87188 in gvBurnScanlineBasic<double> (dfVariant=<optimized out>, nXEnd=<optimized out>, nXStart=<optimized out>, nY=<optimized out>, psInfo=<optimized out>)
at /home/even/gdal/gdal/alg/gdalrasterize.cpp:91
91 for (int iBand = 0; iBand < psInfo->nBands; iBand++)
(gdb) bt
#0 0x00007ffff5f87188 in gvBurnScanlineBasic<double> (dfVariant=<optimized out>, nXEnd=<optimized out>, nXStart=<optimized out>, nY=<optimized out>, psInfo=<optimized out>)
at /home/even/gdal/gdal/alg/gdalrasterize.cpp:91
#1 gvBurnScanline (dfVariant=<optimized out>, nXEnd=5, nXStart=5, nY=-15072, pCBData=0x7fffffffc5a0) at /home/even/gdal/gdal/alg/gdalrasterize.cpp:267
#2 gvBurnScanline (pCBData=pCBData@entry=0x7fffffffc5a0, nY=nY@entry=0, nXStart=<optimized out>, nXEnd=<optimized out>, dfVariant=<optimized out>) at /home/even/gdal/gdal/alg/gdalrasterize.cpp:203
#3 0x00007ffff5fc6793 in GDALdllImageFilledPolygon (nRasterXSize=10, nRasterYSize=nRasterYSize@entry=10, nPartCount=<optimized out>, panPartSize=0xd9eaa0, padfX=0xc5fe70, padfY=0x10afde0,
dfVariant=0x0, pfnScanlineFunc=0x7ffff5f86c40 <gvBurnScanline(void*, int, int, int, double)>, pCBData=0x7fffffffc5a0, bAvoidBurningSamePoints=false) at /home/even/gdal/gdal/alg/llrasterize.cpp:219
#4 0x00007ffff5f89db7 in gv_rasterize_one_shape (pabyChunkBuf=pabyChunkBuf@entry=0xd02580 "", nXOff=nXOff@entry=0, nYOff=nYOff@entry=0, nXSize=<optimized out>, nYSize=nYSize@entry=10,
nBands=nBands@entry=1, eType=GDT_Float64, nPixelSpace=<optimized out>, nLineSpace=80, bAllTouched=<optimized out>, poShape=0x10240e0, eBurnValueType=GDT_Float64, padfBurnValues=<optimized out>,
panBurnValues=0x0, eBurnValueSrc=GBV_UserBurnValue, eMergeAlg=GRMA_Replace, pfnTransformer=0x7ffff5f8f910 <GDALGenImgProjTransform(void*, int, int, double*, double*, double*, int*)>,
pTransformArg=<optimized out>, nBandSpace=<optimized out>) at /home/even/gdal/gdal/alg/gdalrasterize.cpp:734
#5 0x00007ffff5f8bd32 in GDALRasterizeGeometriesInternal (hDS=hDS@entry=0xc60c60, nBandCount=1, panBandList=0xc59be0, nGeomCount=10, pahGeometries=0x10371c0,
pfnTransformer=0x7ffff5f8f910 <GDALGenImgProjTransform(void*, int, int, double*, double*, double*, int*)>, pTransformArg=0x1222ad0, eBurnValueType=GDT_Float64, padfGeomBurnValues=0x0,
panGeomBurnValues=0x0, papszOptions=0x0, pfnProgress=0x7ffff5e2b5a0 <GDALScaledProgress(double, char const*, void*)>, pProgressArg=0x0) at /home/even/gdal/gdal/ogr/ogr_geometry.h:663
#6 0x00007ffff5f8c16e in GDALRasterizeGeometries (hDS=hDS@entry=0xc60c60, nBandCount=<optimized out>, panBandList=<optimized out>, nGeomCount=<optimized out>, pahGeometries=<optimized out>,
pfnTransformer=<optimized out>, pTransformArg=0x0, padfGeomBurnValues=0x0, papszOptions=0x0, pfnProgress=0x7ffff5e2b5a0 <GDALScaledProgress(double, char const*, void*)>, pProgressArg=0x0)
at /home/even/gdal/gdal/alg/gdalrasterize.cpp:970
#7 0x00007ffff6b0a93d in ProcessLayer (hSrcLayer=hSrcLayer@entry=0xc443b0, bSRSIsSet=<optimized out>, hDstDS=hDstDS@entry=0xc60c60, anBandList=std::vector of length 1, capacity 1 = {...},
adfBurnValues=std::vector of length 0, capacity 0, b3D=b3D@entry=false, bInverse=false, osBurnAttribute=..., papszRasterizeOptions=<optimized out>, papszTO=<optimized out>,
pfnProgress=<optimized out>, pProgressData=<optimized out>) at /home/even/gdal/gdal/apps/gdal_rasterize_lib.cpp:773
#8 0x00007ffff6b0b332 in GDALRasterize (pszDest=<optimized out>, pszDest@entry=0x0, hDstDS=hDstDS@entry=0xc60c60, hSrcDataset=hSrcDataset@entry=0xc61180, psOptionsIn=psOptionsIn@entry=0xc531c0,
pbUsageError=pbUsageError@entry=0x7fffffffcbcc) at /usr/include/c++/9/bits/basic_string.h:267