Created attachment 139858 [details] [review]
assembling: Add guards to fpi_assemble_lines()

With the goal of not crashing when we try to malloc MAXINT bytes of RAM.

See https://bugzilla.redhat.com/show_bug.cgi?id=1484812

Created attachment 139859 [details] [review]
vfs5011: Error out when no lines were captured

Which avoids passing zero lines to fpi_assemble_lines()

"gmem.c:130: failed to allocate 18446744073709551612 bytes"

 #3  0x00007fe4f6ef428f in g_log (log_domain=log_domain@entry=0x7fe4f6f3506e "GLib", log_level=log_level@entry=G_LOG_LEVEL_ERROR, format=format@entry=0x7fe4f6f3e610 "%s: failed to allocate %lu bytes") at gmessages.c:1398
 #4  0x00007fe4f6ef2ac4 in g_malloc0 (n_bytes=n_bytes@entry=18446744073709551612) at gmem.c:129
 #5  0x00007fe4f8052020 in median_filter (filtersize=25, size=-1, data=0x0) at assembling.c:309
 #6  fpi_assemble_lines (ctx=ctx@entry=0x7fe4f82ac3c0 <assembling_ctx>, lines=0x0, lines_len=0) at assembling.c:389
 #7  0x00007fe4f805f3db in submit_image (ssm=ssm@entry=0x16c3cba360, data=data@entry=0x16c3cb9cc0) at drivers/vfs5011.c:412

See https://bugzilla.redhat.com/show_bug.cgi?id=1484812

If someone has access to a VFS5011 device.

(In reply to Bastien Nocera from comment #3)
> If someone has access to a VFS5011 device.

I can test it tonight.

(In reply to Vasily Khoruzhick from comment #4)
> (In reply to Bastien Nocera from comment #3)
> > If someone has access to a VFS5011 device.
> 
> I can test it tonight.

Sorry, I can't. VFS5011 in my laptop appears to be broken - it got some dead pixels in the middle and it confuses finger detection in the driver

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/libfprint/libfprint/issues/42.