mat4/mat5: fix int overflow in dataend calculation

The clang sanitizer warns of a possible signed integer overflow when calculating the `dataend` value in `mat4_read_header()`. ``` src/mat4.c:323:41: runtime error: signed integer overflow: 205 * -100663296 cannot be represented in type 'int' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/mat4.c:323:41 in src/mat4.c:323:48: runtime error: signed integer overflow: 838860800 * 4 cannot be represented in type 'int' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/mat4.c:323:48 in ``` Cast the offending `rows` and `cols` ints to `sf_count_t` (the type of `dataend` before performing the calculation, to avoid the issue. CVE: CVE-2022-33065 Fixes: #789 Fixes: #833 Signed-off-by: Alex Stewart <alex.stewart@ni.com>
libsndfile · Oct 20, 2023 · 0754562 · 0754562
1 parent 95201af
commit 0754562
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/src/mat4.c b/src/mat4.c
@@ -320,7 +320,7 @@ mat4_read_header (SF_PRIVATE *psf)
 				psf->filelength - psf->dataoffset, psf->sf.channels * psf->sf.frames * psf->bytewidth) ;
 		}
 	else if ((psf->filelength - psf->dataoffset) > psf->sf.channels * psf->sf.frames * psf->bytewidth)
-		psf->dataend = psf->dataoffset + rows * cols * psf->bytewidth ;
+		psf->dataend = psf->dataoffset + (sf_count_t) rows * (sf_count_t) cols * psf->bytewidth ;
 
 	psf->datalength = psf->filelength - psf->dataoffset - psf->dataend ;