drmModeSetPlane() is just a glorified ioctl(). Errors are negative values, and the error code is in errno.