opencv - Camera pixels to planar world points given 4 known points -

my problem assuming easy, still haven't been able solve due experience in linear algebra while ago. have read presentations published several universities can't seem follow non-standardized notation. if has better example appreciated...

problem: camera angled down facing floor. given pixel coordinate, want able respective 3d world coordinate on plane of floor.

known:

• 4 dots on floor know pixel(x,y) coordinates , associated world(x,y,z=0) coordinates.
• the camera's position fixed , know displacement in x,y,z directions of camera.

unknown:

• the camera's rotations x,y,z axis. primarily, camera rotated x axis y & z being minimal rotations think should taken account.
• distortion coefficients, there minimal bending of images in lines , prefer not bring in checkerboard calibration procedure. error result of not deal breaker.

what i've looked into phenomenal example found here. in essence it's exact same problem, follow questions:

solvepnp looks friend, i'm not sure camera matrix or distcoefficients. there way can avoid camera matrix , dist coefficients calibration steps, done think checkerboard process (maybe @ cost of accuracy)? or there simpler way this?

much appreciate input!

try approach:

compute homography 4 point correspondences, giving information transform between image plane , ground plane coordinates.

limitation of approach assumes uniformly parameterized image plane (pinhole camera), lens distortion give errors seen in example. if able remove lens distortion effects, you'll go approach guess. in addition error of giving wrong pixel coordinates correspondences, can more stable values if provide more correspondences.

using input image

i've read 4 corners of 1 chess field image manipulation software, correspond fact know 4 points in image. i've chosen points (marked green):

now i've done 2 things: first transforming chessboard pattern coordinates image (0,0) , (0,1) etc gives visual impression of mapping quality. second transform image world. reading leftmost corner position in image location (87,291) corresponds (0,0) in chessboard coordinates. if transform pixel location expect (0,0) result.

cv::point2f transformpoint(cv::point2f current, cv::mat transformation) {     cv::point2f transformedpoint;     transformedpoint.x = current.x * transformation.at<double>(0,0) + current.y * transformation.at<double>(0,1) + transformation.at<double>(0,2);     transformedpoint.y = current.x * transformation.at<double>(1,0) + current.y * transformation.at<double>(1,1) + transformation.at<double>(1,2);     float z = current.x * transformation.at<double>(2,0) + current.y * transformation.at<double>(2,1) + transformation.at<double>(2,2);     transformedpoint.x /= z;     transformedpoint.y /= z;      return transformedpoint; }  int main() {     // image http://d20uzhn5szfhj2.cloudfront.net/media/catalog/product/cache/1/image/9df78eab33525d08d6e5fb8d27136e95/5/2/52440-chess-board.jpg      cv::mat chessboard = cv::imread("../inputdata/52440-chess-board.jpg");      // known input:     // image locations / read pixel values     //  478,358     //  570, 325     //  615,382     //  522,417      std::vector<cv::point2f> imagelocs;     imagelocs.push_back(cv::point2f(478,358));     imagelocs.push_back(cv::point2f(570, 325));     imagelocs.push_back(cv::point2f(615,382));     imagelocs.push_back(cv::point2f(522,417));      for(unsigned int i=0; i<imagelocs.size(); ++i)     {         cv::circle(chessboard, imagelocs[i], 5, cv::scalar(0,0,255));     }     cv::imwrite("../outputdata/chessboard_4points.png", chessboard);      // known input: 1 field of chessboard. enter (corresponding) real world coordinates of ground plane here.     // world location:     // 3,3     // 3,4     // 4,4     // 4,3      std::vector<cv::point2f> worldlocs;     worldlocs.push_back(cv::point2f(3,3));     worldlocs.push_back(cv::point2f(3,4));     worldlocs.push_back(cv::point2f(4,4));     worldlocs.push_back(cv::point2f(4,3));       // 4 correspondences. more can use cv::findhomography     // transformation image coordinates world coordinates:     cv::mat image2world = cv::getperspectivetransform(imagelocs, worldlocs);     // inverse transformation world image.     cv::mat world2image = image2world.inv();       // create known locations of chessboard (0,0) (0,1) etc transform them , test how transformation is.     std::vector<cv::point2f> worldlocations;     for(unsigned int i=0; i<9; ++i)         for(unsigned int j=0; j<9; ++j)         {             worldlocations.push_back(cv::point2f(i,j));         }       std::vector<cv::point2f> imagelocations;      for(unsigned int i=0; i<worldlocations.size(); ++i)     {         // transform point         cv::point2f tpoint = transformpoint(worldlocations[i], world2image);         // draw transformed point         cv::circle(chessboard, tpoint, 5, cv::scalar(255,255,0));     }      // test other way: image => world     cv::point2f imageorigin = cv::point2f(87,291);     // draw show origin mean     cv::circle(chessboard, imageorigin, 10, cv::scalar(255,255,255));     // transform point , print result. expected result "(0,0)"     std::cout << transformpoint(imageorigin, image2world) << std::endl;      cv::imshow("chessboard", chessboard);     cv::imwrite("../outputdata/chessboard.png", chessboard);     cv::waitkey(-1);   } 

the resulting image is:

as can see there big amount of error in data. said it's because of wrong pixel coordinates given correspondences (and within small area!) , because of lens distortion preventing ground plane appear real plane on image.

results of transforming (87,291) world coordinates are:

[0.174595, 0.144853] 

expected/perfect result would've been [0,0]

hope helps.