04 image formation 成像过程

image formation 成像过程

Posted on 6 mins read View times

visible light images 可见光信息

Image formation components for visible light images 形成可见光图像的成分

  1. Light source / illumination 光源/照明
  2. Surface geometry 表面几何
  3. Surface reflectance 表面反射
  4. Imaging geometry / optics 成像几何/光学
  5. Sensor characteristics 传感器特性

What is light?

Electromagnetic radiation (EMR) moving along rays in space 空间中沿射线运动的电磁辐射

The visible light spectrum 视觉光谱

We “see” electromagnetic radiation in a range of wavelengths 人眼可见光范围 400-700nm

图 ppt4-8！！！！！！！！！！！！！！！！！！！！！！！！！！！！

Light spectrum 光谱

The appearance of light depends on its power spectrum 光的出现取决于它的功率谱
How much power (or energy) at each wavelength 每个波长有多少功率(或能量)
Our visual system converts a light spectrum into “color” 我们的视觉系统将光谱转换成“颜色”
This is a rather complex transformation 这是一个相当复杂的变换

The human visual system

Color perception Light hits the retina, which contains photosensitive cells 光线照射到视网膜上，视网膜上有感光细胞
These cells convert the spectrum into a few discrete values 这些细胞将光谱转换成几个离散的值

Two types of light-sensitive receptors

Cones 锥状细胞

  1. cone-shaped
  2. less sensitive
  3. operate in high light
  4. color vision
    Rods 杆状细胞
  5. rod-shaped
  6. highly sensitive
  7. operate at night
  8. gray-scale vision

Density of rods and cones:
Rods and cones are non-uniformly distributed on the retina 视杆细胞和视锥细胞在视网膜上呈非均匀分布
Rods responsible for intensity, cones responsible for color 杆状体负责强度，锥状体负责颜色
Fovea - Small region (1 or 2°) at the center of the visual field containing the highest density of cones (and no rods). 中央凹-视野中心的一个小区域(1或2°)，含有最高密度的视锥细胞(没有视杆细胞)。 Less visual acuity in the periphery—many rods wired to the same neuron 其他区域的视敏度较低——许多杆状体连接到同一个神经元

Brightness contrast and constancy 亮度对比度和稳定性

The apparent brightness depends on the surrounding region 视亮度取决于周围的区域
brightness contrast 亮度对比: a constant colored region seem lighter or darker depending on the surround 一个固定颜色的区域根据周围环境的不同会看起来更亮或更暗
brightness constancy 稳定性: a surface looks the same under widely varying lighting conditions.在大范围变化的光照条件下，表面看起来是一样的

Light response is nonlinear 光的响应是非线性的

Our visual system has a large dynamic range 我们的视觉系统有很大的动态范围
We can resolve both light and dark things at the same time 我们可以同时解决光明和黑暗的问题
One mechanism for achieving this is that we sense light intensity on a logarithmic scale 一种方法是，我们在对数标度上感知光强
an exponential intensity ramp will be seen as a linear ramp 指数强度斜坡将被视为线性下降
Another mechanism is adaptation 另一种机制是适应
rods and cones adapt to be more sensitive in low light, less sensitive in bright light. 杆状体和锥状体在弱光下更敏感，在强光下更不敏感

Color perception 颜色感知

Three types of cones 三种锥状细胞 Each is sensitive in a different region of the spectrum 每一个在光谱的不同区域都是敏感的
but regions overlap
Short (S) corresponds to blue
Medium (M) corresponds to green
Long (L) corresponds to red
Different sensitivities: we are more sensitive to green than red 不同的敏感度:我们对绿色比红色更敏感
varies from person to person (and with age)
Colorblindness 色盲 —deficiency in at least one type of cone 至少有一种锥体色盲
Rods and cones act as filters on the spectrum 杆状体和锥状体在光谱上起过滤作用
To get the output of a filter, multiply its response curve by the spectrum, integrate over all wavelengths 要得到滤波器的输出的话，要将其响应曲线乘以光谱，然后对所有波长积分
Q: How can we represent an entire spectrum with 3 numbers? 我们如何用3个数字表示整个频谱?
A: We can’t! Most of the information is lost. 不能 大部分信息都丢失了

Practical Color Sensing: Bayer Grid

Each pixel senses light in one color band only 每个像素只能感知一个色带中的光
The highest sampling is in the green channel 绿色通道的采样率最高
There is close proximity of each pixel color to the other two colors (due to the arrangement of the color filters) 每个像素的颜色与其他两种颜色非常接近(由于颜色过滤器的安排)

Digital camera

The sensor array elements sense the amount of light hitting them, and they only sense the amount of light in only one color. But we need to produce the light contribution of all three colors for each pixel.
This is called the process of “demosaicing”去马赛克

Color representation and color spaces 颜色空间

Color spaces: RGB

Some drawbacks

  1. Strongly correlated channels 相关性很强，不方便调整
  2. Non-perceptual

Color spaces: HSV

Intuitive color space 直观的彩色空间

  1. 色调(Hue)：角度
  2. 饱和度(Saturation)：中心轴到外部的距离表示饱和度。半径越长（越靠外），饱和度越高，看上去颜色更浓
  3. 明度(Value)：从底下到最上面，颜色从最暗到最亮

图 ppt4-30!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Color spaces: YCbCr

亮度和色差信号

  1. Fast to compute 快速计算
  2. good for compression 有利于压缩
  3. used by analog TV (component colors) 用于模拟电视(组件颜色)
    Y'为颜色的亮度(luma)成分、而CB和CR则为蓝色和红色的浓度偏移量成份
    Y分量最重要，除了只有灰度颜色，其他信息是完整的 CB和CR是色差信息
>Y' = Kr * R'+ （1 - Kr - Kb) * G' + Kb * B'
>Pb = 0.5 * (B' - Y') / （1 - Kb)
>Pr = 0.5 * (R' - Y') / （1 - Kr)

彩色度变灰度图：之前用公式求Y分量就好

Color spaces: L* a* b*

不同频率不同幅度的正弦波的叠加，得到的频谱是用来组成这个频率信号的每一个分量的幅度。
L：明度（Luminosity），值域0~100，L=50相当于50%的黑
A：洋红~绿色 +127~-128
B：黄色~蓝色 +127~-128
特点：色域宽阔，能表现RGB，CMYK不能表现的色彩。人的肉眼能感知的色 彩，都能通过Lab模型表现出来。
弥补了RGB色彩模型色彩分布不均的不足，RGB模型在蓝色到绿色之间的过渡色彩过多，而在绿色到红色之间又缺少黄色和其他色彩。

Light sources 光源

  1. Point light source (nearby) 点光源
  2. （默认）Directional light sources: 定向光源
    a point source “infinitely” far away (sun): only the light direction matters 一个“无限”远的点源(太阳):只有光的方向是重要的
  3. Extended light sources 扩展光源:
    Union of point or directional light sources (fluorescent lamps) 点光源或方向光源的组合(荧光灯)

Surface types 表面种类

How the light that hits a surface gets reflected determines what we see 光线照射到表面时如何反射决定了我们所看到的东西
There are many different types of surfaces

The interaction of light and matter 光与物质的相互作用

What happens when a light ray hits a point on an object? 当光线照射到物体上的一点时会发生什么？

  1. Some of the light gets absorbed 有些光被吸收了
    converted to other forms of energy (e.g., heat) 转化为其他形式的能量(如热)
  2. Some gets transmitted through the object 有些通过物质传输
    possibly bent, through “refraction” 可能弯曲，通过“折射”
  3. Some gets reflected 一些被反射
    it could be reflected in multiple directions at once 它可以同时多个方向反射

A photon’s life choices 光子 选择

  1. Absorption 吸收
  2. Diffusion 扩散
    Effect is that light is reflected equally in all directions Lambertian surface.（均匀漫反射面） Diffuse reflection governed by Lambert’s law 漫反射受朗伯定律支配 Viewed brightness does not depend on viewing direction 观看的亮度跟方向无关，都一样 Brightness does depend on direction of illumination 亮度跟入射方向有关，角度不同反射的能量不同，越垂直能量越高亮度越大 This is the model most often used in computer vision

图 ppt4-52！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！

  1. Reflection 反射
  2. Transparency 透明度
  3. Refraction 折射
  4. Fluorescence 荧光

三种光照方式

  1. 环境光照(Ambient Lighting)：即使在黑暗的情况下，世界上通常也仍然有一些光亮（月亮、远处的光），所以物体几乎永远不会是完全黑暗的。为了模拟这个，我们会使用一个环境光照常量，它永远会给物体一些颜色。
  2. 漫反射光照(Diffuse Lighting)：模拟光源对物体的方向性影响(Directional Impact)。它是冯氏光照模型中视觉上最显著的分量。物体的某一部分越是正对着光源，它就会越亮。
  3. 镜面光照(Specular Lighting)：模拟有光泽物体上面出现的亮点。镜面光照的颜色相比于物体的颜色会更倾向于光的颜色。

Modeling perspective projection 透视成像

Projective Geometry 投影几何

Straight lines are still straight 角度长度平行性垂直性都不保留，只有直线变换后还是直线

Pinhole camera model basics

Normally, the image plane is behind the center of projection (pinhole) and the real image formed is upside down. 通常情况下，成像平面位于投影中心(针孔)后方，形成的真实图像是上下颠倒的。
Computer Vision cameras are modeled as the image plane in front of center of projection 将计算机视觉摄像机建模为投影中心前的像平面
so images are not upside down. 所以图像不是颠倒的。

Camera calibration for pinhole camera model 小孔成像的相机标定

Given a camera, how do we estimate its specific parameters?

  1. Intrinsic parameters: focal length, scale factor, image center, etc. 内部：焦距、比例因子、图像中心等
  2. Extrinsic parameters: pose (rotation and translation) 外部：形态(旋转、平移)

实验1！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！！