Duplicate

CV_HW3 (writeup)

마감일
2022/11/07
제출자
전기정보공학부 2017-12433 김현우

Part 1 Estimating transformations from the image points

H=compute_h(p1,p2)\rm H = compute\_h(p1, p2)
From corresponding points p1\rm p1, p2\rm p2, p1=Hp2\rm p_1 = Hp_2 is satisfied where p1∈p1,p2∈p2p_1\in p1, p_2\in p2. Thus, using these corresponding points, I modify this equation as Ah=0Ah=0 form by constructing matrix AA and linearize matrix H\rm H as 1 dimensional vector hh.
After obtaining matrix AA, I applied SVD to get eigenvector which has smallest eigenvalue to make ∥Ah∥\|Ah\| minimum and assign the eigenvetor as hh.
Then, I reshape 1 dimensional vector hh to be 3×33\times 3 matrix to get final homography H\rm H we want to compute.
H=compute_h_norm(p1,p2)\rm H = compute\_h\_norm(p1, p2)
p1\rm p1 and p2\rm p2 are generally obtained from different image. Thus for each data, I find out maximum value of xx and yy pixels. Then I divide each xx pixel data to each maximum xx value and yy pixel data to each maximum yy value to normalize the pixel value into [0,1][0,1].
Then, I applied compute_h\rm compute\_h fuction with normalized version of p1\rm p1 and p2\rm p2. This outputs the homogrpahy Hnorm\rm H_{norm} between normalized p1\rm p1 and p2\rm p2 stably.
To make sure the output homography to be same as the original homography, I have to use diagonal matrix which is composed with maximum value of xx, maximum value of yy, and 1 as diagonal term.
p1=Hp2→[p1xp1y1]=H[p2xp2y1]→[max⁡(p1x)000max⁡(p1y)1001][p1x/max⁡(p1x)p1y/max⁡(p1y)1]=H[max⁡(p2x)000max⁡(p2y)1001][p2x/max⁡(p2x)p2y/max⁡(p2y)1]\begin{align*} \rm p_1 = H p_2 &\rarr \begin{bmatrix} p_{1x}\\ p_{1y}\\ 1 \end{bmatrix} = {\rm{H}} \begin{bmatrix} p_{2x}\\ p_{2y}\\ 1 \end{bmatrix} \\ &\rarr \begin{bmatrix} \max(p_{1x}) & 0 & 0\\ 0 &\max(p_{1y}) & 1\\ 0 & 0 &1 \end{bmatrix} \begin{bmatrix} p_{1x} / \max(p_{1x})\\ p_{1y} / \max(p_{1y})\\ 1 \end{bmatrix}\\ &= {\rm{H}} \begin{bmatrix} \max(p_{2x}) & 0 & 0\\ 0 &\max(p_{2y}) & 1\\ 0 & 0 &1 \end{bmatrix} \begin{bmatrix} p_{2x} / \max(p_{2x})\\ p_{2y} / \max(p_{2y})\\ 1 \end{bmatrix} \\ \end{align*}
Above equation expresses what I’ve done in implementation. I have to finally figure out H\rm H using normalized vectors.
[p1x/max⁡(p1x)p1y/max⁡(p1y)1]=[max⁡(p1x)000max⁡(p1y)1001]−1H[max⁡(p2x)000max⁡(p2y)1001][p2x/max⁡(p2x)p2y/max⁡(p2y)1] \begin{bmatrix} p_{1x} / \max(p_{1x})\\ p_{1y} / \max(p_{1y})\\ 1 \end{bmatrix} = \begin{bmatrix} \max(p_{1x}) & 0 & 0\\ 0 &\max(p_{1y}) & 1\\ 0 & 0 &1 \end{bmatrix}^{-1} {\rm{H}} \begin{bmatrix} \max(p_{2x}) & 0 & 0\\ 0 &\max(p_{2y}) & 1\\ 0 & 0 &1 \end{bmatrix} \begin{bmatrix} p_{2x} / \max(p_{2x})\\ p_{2y} / \max(p_{2y})\\ 1 \end{bmatrix} \\
By above equation I can think of below relationship.
Hnorm=[max⁡(p1x)000max⁡(p1y)1001]−1H[max⁡(p2x)000max⁡(p2y)1001][p2x/max⁡(p2x)p2y/max⁡(p2y)1]{\rm H_{norm}} = \begin{bmatrix} \max(p_{1x}) & 0 & 0\\ 0 &\max(p_{1y}) & 1\\ 0 & 0 &1 \end{bmatrix}^{-1} {\rm{H}} \begin{bmatrix} \max(p_{2x}) & 0 & 0\\ 0 &\max(p_{2y}) & 1\\ 0 & 0 &1 \end{bmatrix} \begin{bmatrix} p_{2x} / \max(p_{2x})\\ p_{2y} / \max(p_{2y})\\ 1 \end{bmatrix}
Then, I can finally derive that H\rm H can be computed by left-multiplying
[max⁡(p1x)000max⁡(p1y)1001]\begin{bmatrix} \max(p_{1x}) & 0 & 0\\ 0 &\max(p_{1y}) & 1\\ 0 & 0 &1 \end{bmatrix} and right-multiplying[max⁡(p2x)000max⁡(p2y)1001]−1\begin{bmatrix} \max(p_{2x}) & 0 & 0\\ 0 &\max(p_{2y}) & 1\\ 0 & 0 &1 \end{bmatrix}^{-1}to Hnorm\rm H_{norm}.
Thus, I can finally compute the H\rm H using Hnorm\rm H_{norm} computed from normalized vectors.

Part 2 Mosaicing

pin,pref=set_cor_mosaic()\rm p_{in}, p_{ref} = set\_cor\_mosaic()
I manually find out corresponding points by visualizing image with matplotlib imshow()\rm imshow() and extract pixel points by mouseover-ing the interesting points in the visualization.
igs_warp,igs_merge=warp_image(igs_in,igs_ref,H)\rm igs\_warp, igs\_merge = warp\_image(igs\_in, igs\_ref, H)
To avoid holes in image, I used inverse warping. First I compute the inverse homography using np.linalg.inv()\rm np.linalg.inv(). Then, I firstly defined warped image same size with reference image, and iterate over the pixel and apply inverse homogrpaphy to each pixel. This maps the points to original image, thus I find out the mapped pixel on original image.
In general, the mapped pixel on original image consists of floating number, thus I used bilinear interpolation with nearest 4 pixels. I can finally get corresponding original pixel value and successfully paint the region in warped image. If the mapped pixel on original image is out of range, I just paint it as 0 (black).
After implementing, I found out the warped image which has same size with reference image doesn’t show all of warped image regions. Thus, I add padding on left, right, up, and down to warped image to safely containing the computed value which has negative or much bigger than reference image size.

Part 3 Rectification

cin,cref=set_cor_rec()\rm c_{in}, c_{ref} = set\_cor\_rec()
For setting c_in\rm c\_in, I manually find out 4 corner points pixels of iphone screen by visualizing image with matplotlib imshow()\rm imshow() and extract pixel points by mouseover-ing the points in the visualization.
For setting c_ref\rm c\_ref, I find out the reference that ratio of width and height of recent iphones are 9:19.59:19.5. Thus I set 4 manual pixels consisting rectangle on image, which width and height are parallel to image width and image and has ratio of 9:19.59:19.5.
igs_rec=rectify(igs,p1,p2)\rm igs\_rec = rectify(igs, p1, p2)
This implementation is totally similar with the warped_image implementing part in warp_image(igs_in,igs_ref,H)\rm warp\_image(igs\_in, igs\_ref, H). Initial size of the warped image is changed in this function, and I have to call compute_h_norm\rm compute\_h\_norm manually in this function because there are no homography argument passed in this function.
The leftover implementation is same as warp_image(igs_in,igs_ref,H)\rm warp\_image(igs\_in, igs\_ref, H). I computed inverse homography, and implement inverse warping by iterating warped image pixel and apply inverse homogrpahy to find out mapped original pixel. Then, I find out each mapped original pixel value using bilinear interpolation, too.