[argparse][gdal_rasterize] Port to arg parser #10741

elpaso · 2024-09-06T14:29:32Z

Usage: gdal_rasterize [--help] [--long-usage] [--help-general]
                      [-b <band>...]... [-i] [-at]
                      [[-burn <value>...]...|[-a <attribute_name>]|[-3d]]
                      [-add]
                      [[-l <layer_name>]...|[-sql <sql_statement>]]
                      [-where <expression>] [-dialect <sql_dialect>] [-a_nodata <value>] [-init <value>...]...
                      [-a_srs <srs_def>] [-to <NAME>=<VALUE>]... [-te <xmin> <ymin> <xmax> <ymax>]
                      [[-tr <xres> <yres>]|[-ts <width> <height>]]
                      [-tap] [-optim AUTO|VECTOR|RASTER] [-co <NAME>=<VALUE>]...
                      [-ot Byte|Int8|[U]Int{16|32|64}|CInt{16|32}|[C]Float{32|64}] [-of <output_format>]
                      [-oo <NAME>=<VALUE>]... [--quiet]
                      <src_datasource> <dst_filename>

Burns vector geometries into a raster.

Positional arguments:
  <src_datasource>                                            Any vector supported readable datasource. 
  <dst_filename>                                              The GDAL raster supported output file. 

Optional arguments:
  -h, --help                                                  Shows short help message and exits. 
  --long-usage                                                Shows long help message and exits. 
  --help-general                                              Report detailed help on general options. 
  -b                                                          The band(s) to burn values into. [nargs: 1 or more] [may be repeated]
  -i                                                          Invert rasterization. 
  -at                                                         Enables the ALL_TOUCHED rasterization option. 
  -burn                                                       A fixed value to burn into the raster band(s). [nargs: 1 or more] [may be repeated]
  -a <attribute_name>                                         Name of the field in the input layer to get the burn values from. 
  -3d                                                         Indicates that a burn value should be extracted from the "Z" values of the feature. 
  -add                                                        Instead of burning a new value, this adds the new value to the existing raster. 
  -l <layer_name>                                             Name of the layer(s) to process. [may be repeated]
  -sql <sql_statement>                                        An SQL statement to be evaluated against the datasource to produce a virtual layer of features to be burned in. 
  -where <expression>                                         An optional SQL WHERE style query expression to be applied to select features to burn in from the input layer(s). 
  -dialect <sql_dialect>                                      The SQL dialect to use for the SQL expression. 
  -a_nodata <value>                                           Assign a specified nodata value to output bands. 
  -init                                                       Initialize the output bands to the specified value. [nargs: 1 or more] [may be repeated]
  -a_srs <srs_def>                                            The spatial reference system to use for the output raster. 
  -to <NAME>=<VALUE>                                          Set a transformer option. [may be repeated]
  -te <xmin> <ymin> <xmax> <ymax>                             Set georeferenced extents of output file to be created. [nargs: 4] 
  -tr <xres> <yres>                                           Set output file resolution in target georeferenced units. [nargs: 2] 
  -ts <width> <height>                                        Set output file size in pixels and lines. [nargs: 2] 
  -tap                                                        Align the coordinates of the extent to the values of the output raster. 
  -optim AUTO|VECTOR|RASTER                                   Force the algorithm used. 
  -co <NAME>=<VALUE>                                          Creation option(s). [may be repeated]
  -ot Byte|Int8|[U]Int{16|32|64}|CInt{16|32}|[C]Float{32|64}  Output data type. 
  -of <output_format>                                         Output format. 
  -oo <NAME>=<VALUE>                                          Open option(s) for input dataset. [may be repeated]
  -q, --quiet                                                 Quiet mode. No progress message is emitted on the standard output. 

This program burns vector geometries (points, lines, and polygons) into the raster band(s) of a raster image.

apps/gdal_rasterize_lib.cpp

Co-authored-by: Even Rouault <even.rouault@spatialys.com>

coveralls · 2024-09-06T16:28:28Z

coverage: 69.343% (+0.003%) from 69.34%
when pulling 4be1205 on elpaso:argparse-gdal_rasterize
into 8f4571f on OSGeo:master.

tbonfort · 2024-09-11T10:22:35Z

I'm getting a segmentation fault since (I believe) this update, when using -burn 0 on a 3 band dataset

Thread 1 "godal.test" received signal SIGSEGV, Segmentation fault.
0x00007ffff5ef6207 in gvBurnScanlineBasic<unsigned char> (psInfo=0x7fffffffd720, nY=1, nXStart=1, nXEnd=1, dfVariant=0)
    at /home/tbonfort/thirdparty/gdal/alg/gdalrasterize.cpp:94
94	            (psInfo->burnValues.double_values[iBand] +
(gdb) print psInfo->burnValues.double_values[0]
Cannot access memory at address 0x0
(gdb) bt
#0  0x00007ffff5ef6207 in gvBurnScanlineBasic<unsigned char> (psInfo=0x7fffffffd720, nY=1, nXStart=1, nXEnd=1, dfVariant=0)
    at /home/tbonfort/thirdparty/gdal/alg/gdalrasterize.cpp:94
#1  0x00007ffff5ef176c in gvBurnScanline (pCBData=0x7fffffffd720, nY=1, nXStart=1, nXEnd=1, dfVariant=0)
    at /home/tbonfort/thirdparty/gdal/alg/gdalrasterize.cpp:238
#2  0x00007ffff5f5c16d in GDALdllImageFilledPolygon (nRasterXSize=3, nRasterYSize=3, nPartCount=1, panPartSize=0x1591fc0, 
    padfX=0x16740a0, padfY=0x1671b10, dfVariant=0x0, pfnScanlineFunc=0x7ffff5ef15e1 <gvBurnScanline(void*, int, int, int, double)>, 
    pCBData=0x7fffffffd720, bAvoidBurningSamePoints=false) at /home/tbonfort/thirdparty/gdal/alg/llrasterize.cpp:219
#3  0x00007ffff5ef2ed4 in gv_rasterize_one_shape (pabyChunkBuf=0x15ca0e0 "\377\377\377\377\377\377\377\377\377", nXOff=0, nYOff=0, 
    nXSize=3, nYSize=3, nBands=1, eType=GDT_Byte, nPixelSpace=1, nLineSpace=3, nBandSpace=9, bAllTouched=0, poShape=0x18af490, 
    eBurnValueType=GDT_Float64, padfBurnValues=0x0, panBurnValues=0x0, eBurnValueSrc=GBV_UserBurnValue, eMergeAlg=GRMA_Replace, 
    pfnTransformer=0x7ffff5f0abe3 <GDALGenImgProjTransform(void*, int, int, double*, double*, double*, int*)>, pTransformArg=0x1674eb0)
    at /home/tbonfort/thirdparty/gdal/alg/gdalrasterize.cpp:734
#4  0x00007ffff5ef3c18 in GDALRasterizeGeometriesInternal (hDS=0x15d7460, nBandCount=1, panBandList=0x1676560, nGeomCount=2, 
    pahGeometries=0x187c290, 
    pfnTransformer=0x7ffff5f0abe3 <GDALGenImgProjTransform(void*, int, int, double*, double*, double*, int*)>, 
    pTransformArg=0x1674eb0, eBurnValueType=GDT_Float64, padfGeomBurnValues=0x0, panGeomBurnValues=0x0, papszOptions=0x0, 
    pfnProgress=0x7ffff5c9c0c2 <GDALScaledProgress(double, char const*, void*)>, pProgressArg=0x0)
    at /home/tbonfort/thirdparty/gdal/alg/gdalrasterize.cpp:1199
#5  0x00007ffff5ef32c6 in GDALRasterizeGeometries (hDS=0x15d7460, nBandCount=1, panBandList=0x1676560, nGeomCount=2, 
    pahGeometries=0x187c290, pfnTransformer=0x0, pTransformArg=0x0, padfGeomBurnValues=0x0, papszOptions=0x0, 
    pfnProgress=0x7ffff5c9c0c2 <GDALScaledProgress(double, char const*, void*)>, pProgressArg=0x0)
    at /home/tbonfort/thirdparty/gdal/alg/gdalrasterize.cpp:970
#6  0x00007ffff74bc09f in ProcessLayer (hSrcLayer=0x1594170, bSRSIsSet=false, hDstDS=0x15d7460, 
    anBandList=std::vector of length 1, capacity 1 = {...}, adfBurnValues=std::vector of length 0, capacity 0, b3D=false, 
    bInverse=false, osBurnAttribute="", papszRasterizeOptions=0x0, papszTO=0x0, 
    pfnProgress=0x7ffff5c9c0c2 <GDALScaledProgress(double, char const*, void*)>, pProgressData=0x0)
    at /home/tbonfort/thirdparty/gdal/apps/gdal_rasterize_lib.cpp:773
#7  0x00007ffff74bdd3c in GDALRasterize (pszDest=0x15d7480 "", hDstDS=0x15d7460, hSrcDataset=0x1593c60, psOptionsIn=0x164b9b0, 
    pbUsageError=0x7fffffffdea4) at /home/tbonfort/thirdparty/gdal/apps/gdal_rasterize_lib.cpp:1257

rouault · 2024-09-11T10:42:41Z

I'm getting a segmentation fault since (I believe) this update, when using -burn 0 on a 3 band dataset

@elpaso Can you have a look at that ?

elpaso · 2024-09-11T11:06:45Z

@tbonfort I cannot reproduce, can you please provide a small data sample and the command line to reproduce the crash?

tbonfort · 2024-09-11T11:24:20Z

@elpaso there is no sample data per se, this is raised during the CI in https://github.com/airbusgeo/godal/blob/203295be1c72687019cbe7774ea02eb0c84fe489/godal_test.go#L2405

rouault · 2024-09-11T12:38:34Z

@tbonfort I've tried to replicate your go unit test with the following, but it works fine for me

$ wget https://raw.githubusercontent.com/airbusgeo/godal/main/testdata/test.geojson
$ gdal_create -outsize 3 3 -a_srs EPSG:4326 -a_ullr 99.1 -0.9 102.1 1.9 -burn 255 out.tif
$ PYTHONMALLOC=malloc valgrind  python -c "from osgeo import gdal; src_ds = gdal.OpenEx('test.geojson'); out_ds = gdal.OpenEx('out.tif', gdal.OF_UPDATE); gdal.Rasterize(out_ds, src_ds, options = '-burn 0')"
$ gdal_translate out.tif /vsistdout/ -of AAIGRID -b 1
[...]
255 255 255 
255 0 255 
255 255 255

tbonfort · 2024-09-11T13:05:09Z

It seems to be the repetition of using the/an arg parser that causes the issue. I finally managed to reproduce the segfault with

#include <gdal.h>
#include <gdal_utils.h>
#include <ogr_srs_api.h>

int main(int argc, char **argv) {
    GDALAllRegister();

    GDALDatasetH vec = GDALOpenEx("GEOJSON:{\"type\":\"Feature\",\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[0,0],[0,1],[1,1],[1,0],[0,0]]]}}", GDAL_OF_VECTOR, NULL, NULL, NULL);

    GDALDriverH memd = GDALGetDriverByName("MEM");
    GDALDatasetH memds = GDALCreate(memd,"",3,3,3,GDT_Byte,NULL);
    double gt[] = { 0, 1, 0, 1, 0, -1 };
    GDALSetGeoTransform(memds,gt);
    for(int i=1; i<=3; i++) {
        GDALFillRaster(GDALGetRasterBand(memds,i),255,0);
    }
	OGRSpatialReferenceH sr = OSRNewSpatialReference(NULL);
	OSRSetAxisMappingStrategy(sr, OAMS_TRADITIONAL_GIS_ORDER);
	OSRImportFromEPSG(sr, 4326);
    GDALSetSpatialRef(memds,sr);

    char *switches[] = {NULL};
    GDALRasterizeOptions *ropts = GDALRasterizeOptionsNew(switches,NULL);
    int rerr;
    GDALRasterize(NULL,memds,vec,ropts,&rerr);
    
    char *switches2[] = {"-burn","0",NULL};
    ropts = GDALRasterizeOptionsNew(switches2,NULL);
    GDALRasterize(NULL,memds,vec,ropts,&rerr);

}```

rouault · 2024-09-11T13:18:03Z

Actually it seems it is the lack of any command line argument, since I reproduce the crash with:

python -c "from osgeo import gdal; src_ds = gdal.OpenEx('test.geojson'); out_ds = gdal.OpenEx('out.tif', gdal.OF_UPDATE); gdal.Rasterize(out_ds, src_ds);"

tbonfort · 2024-09-11T14:28:47Z

Correct, it's the empty arg list alone that's causing the issue.
A minimal C reproducer without any data dependency can therefore be:

#include <gdal.h>
#include <gdal_utils.h>

int main(int argc, char **argv) {
    GDALAllRegister();

    GDALDatasetH vec = GDALOpenEx("GEOJSON:{\"type\":\"Feature\",\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[0,0],[0,1],[1,1],[1,0],[0,0]]]}}", GDAL_OF_VECTOR, NULL, NULL, NULL);

    GDALDriverH memd = GDALGetDriverByName("MEM");
    GDALDatasetH memds = GDALCreate(memd,"",3,3,3,GDT_Byte,NULL);

    char *switches[] = {NULL};
    GDALRasterizeOptions *ropts = GDALRasterizeOptionsNew(switches,NULL);
    int rerr;
    GDALRasterize(NULL,memds,vec,ropts,&rerr);
}

Followup OSGeo#10741

…master only) Fixes #10767 Followup #10741

[argparse][gdal_rasterize] Port to arg parser

fd387ba

elpaso added the enhancement label Sep 6, 2024

cast

a718025

elpaso mentioned this pull request Sep 6, 2024

gdal_rasterize docs: add notes about -at and -sql options #10726

Merged

5 tasks

rouault reviewed Sep 6, 2024

View reviewed changes

elpaso and others added 3 commits September 6, 2024 17:03

Apply suggestions from code review

5c424b5

Co-authored-by: Even Rouault <even.rouault@spatialys.com>

no mutex

d436cb8

fix init GDALRasterizeOptions

4be1205

rouault added this to the 3.10.0 milestone Sep 6, 2024

rouault merged commit 8d9ceab into OSGeo:master Sep 6, 2024
35 checks passed

tbonfort mentioned this pull request Sep 11, 2024

HasVSIHandler to test if a VSIHandler is registered for a given prefix airbusgeo/godal#133

Merged

rouault mentioned this pull request Sep 11, 2024

gdal_rasterize: crash when rasterizing on existing file with gdal.Rasterize() API without any option (3.10.0dev only) #10767

Closed

elpaso added a commit to elpaso/gdal that referenced this pull request Sep 11, 2024

[gdal_rasterize] Fix crash with no options after argparser

1586cdc

Followup OSGeo#10741

elpaso mentioned this pull request Sep 11, 2024

[gdal_rasterize] Fix crash with no options after argparser #10770

Merged

rouault pushed a commit that referenced this pull request Sep 11, 2024

[gdal_rasterize] Fix crash with no options after argparser (#10770) (…

5999d8b

…master only) Fixes #10767 Followup #10741

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[argparse][gdal_rasterize] Port to arg parser #10741

[argparse][gdal_rasterize] Port to arg parser #10741

elpaso commented Sep 6, 2024 •

edited

Loading

coveralls commented Sep 6, 2024

tbonfort commented Sep 11, 2024 •

edited

Loading

rouault commented Sep 11, 2024

elpaso commented Sep 11, 2024

tbonfort commented Sep 11, 2024

rouault commented Sep 11, 2024

tbonfort commented Sep 11, 2024

rouault commented Sep 11, 2024

tbonfort commented Sep 11, 2024 •

edited

Loading

[argparse][gdal_rasterize] Port to arg parser #10741

[argparse][gdal_rasterize] Port to arg parser #10741

Conversation

elpaso commented Sep 6, 2024 • edited Loading

coveralls commented Sep 6, 2024

tbonfort commented Sep 11, 2024 • edited Loading

rouault commented Sep 11, 2024

elpaso commented Sep 11, 2024

tbonfort commented Sep 11, 2024

rouault commented Sep 11, 2024

tbonfort commented Sep 11, 2024

rouault commented Sep 11, 2024

tbonfort commented Sep 11, 2024 • edited Loading

elpaso commented Sep 6, 2024 •

edited

Loading

tbonfort commented Sep 11, 2024 •

edited

Loading

tbonfort commented Sep 11, 2024 •

edited

Loading