其他分享
首页 > 其他分享> > 全志T507平台增加V4L2 sensor直接操作I2C的API

全志T507平台增加V4L2 sensor直接操作I2C的API

作者:互联网

大纲

参考

我非常严重的参考了以下的文章,提前列出来:
非常好的文章
Linux V4L2 draft
v4l2的学习建议和流程解析
V4L2框架解析
Linux V4L2 draft
Linux V4L2 draft

作者: 嵌入式Max
V4L2框架全解析

00 - V4L2框架概述
01 - V4L2框架-v4l2 device
02 - V4L2框架-media-device
03 - V4L2框架-videobuf2
V4L2框架-control
V4L2框架-control的数据结构

作者: Letcos
[Camera]v4l2框架内核空间解析

下面的文章作为一般参考,这里记录一下:
V4L2源代码之旅一:struct v4l2_subdev
V4L2源代码之旅二:V4L2 sub-device userspace API
V4L2源代码之旅三:I2C sub-device drivers
V4L2源代码之旅四:struct video_device
V4L2源代码之旅五:V4L2的起点和终点
V4L2源代码之旅六:源码追踪
V4L2源代码之旅七:controls queryctrl()
V4L2源代码之旅八:ioctl
V4L2源代码之旅九:videobuf
V4L2源代码之旅十:videobuf主要结构体

Xilinx Linux V4L2视频管道(Video Pipeline)驱动程序分析

目标

对一个封装为V4L2的Sensor,在AP层增加一个直接操作 I2C的API.

背景:

这里使用的是全志T507的SDK,linux版本是4.9.191

因为在全志平台,sensor设备并不是默认做为一个标准的I2C 设备,可以通过 /dev/i2c-x 可以直接对设备进行读写.而是注册为V4L2的设备( /dev/videox). 这样AP层就无法通过常规的I2C操作来操作Sensor 的 Register.

全志平台的I2C有自己的标准 :TWI : Normal Two Wire Interface :“全志平台兼容I2C标准协议的总线控制协议”.

但是软件层面,用的还是I2C的描述 ,这样就太恶心了,我认为完全没有必要新发明一套私有的体系,这样违背了设立I2C的初衷.

这样做的目的是什么?

之所以注册为/dev/videox,就是为了避免AP层对sensor直接进行I2C操作而可能引发问题.
而I2C的操作,全志自己封装了1层cci来进行操作.

那么具体是使用cci还是使用twi(也就是I2C),从kernel里面的配置可以得知是可以选择的,默认是保留现在的cci方式,可选是将V4L2做为一个常规的I2C设备.

.config - Linux/arm64 4.9.191 Kernel Configuration
Device Drivers > Multimedia support > V4L platform devices > select cci or cci to twi (use internal cci) —>
(X) use internal cci
( ) chenge cci to twi

怎么做?

一开始我也不知道V4L2的命令是如何运作的,那么只能通过解剖麻雀的方法进行研究.
我对subdevice的理解是设备可以作为I2C, sensor, v4l2的子设备,可以互相指向和转化,所以叫做subdevice,这个有新的理解再补充.

// 从v4l2_subdev获取i2c_client:
struct i2c_client *client = v4l2_get_subdevdata(sd);

// 从i2c_client获取v4l2_subdev:
 struct v4l2_subdev *sd = i2c_get_clientdata(client);
 
 
## 从sd成员来获取父成员的地址
struct v4l2_subdev *sd;
struct sensor_info *info = to_state(sd);

## 获取info的成员成员sd的地址
struct sensor_info *info;
struct v4l2_subdev *sd = &info->sd;
## 这2个是一个互为 相反的操作

1 研究一下V4L2的sub device的初始化

在sunxi平台, /dev/video0 的设备,大致是经过下面的流程进行了初始化.

module_init(vin_init);
└──>vin_init(void);
	│	// drivers/media/platform/sunxi-vin/vin-video/vin_core.c
	├──	sunxi_csi_platform_register();
	└──>sunxi_vin_core_register_driver();
		└──>platform_driver_register(&vin_core_driver);
			└──>static struct platform_driver vin_core_driver = {
					.probe = vin_core_probe,
					...
					}
				vin_core_probe();
				│	//drivers/media/platform/sunxi-vin/vin-video/vin_video.c
				└──>vin_initialize_capture_subdev();
					│	// drivers/media/v4l2-core/v4l2-subdev.c				
					│	// 此时将device作为链表加入init序列,并开始初始化具体的device,下面是间接的调用				
					├──>v4l2_subdev_init();
					│ sd->internal_ops = &vin_capture_sd_internal_ops;
					│ 	struct v4l2_subdev_internal_ops vin_capture_sd_internal_ops = {
					│ 	.registered = vin_capture_subdev_registered,
					│	.unregistered = vin_capture_subdev_unregistered,
					│};
					└──v4l2_set_subdevdata(sd)
						└──>vin_capture_subdev_registered()
							| // drivers/media/platform/sunxi-vin/vin-video/vin_video.c
							├──>vin_init_controls()
							│	│ //将3A之类的命令注册一下,并给初始值
							│	├──>v4l2_ctrl_new_std(V4L2_CID_XXX)
							│	├──>v4l2_ctrl_new_std_menu(V4L2_CID_XXX)
							│	└──>v4l2_ctrl_new_custom(V4L2_CID_XXX)
							└──>vin_init_video()
								├──>cap->vdev.name // vin_video0
								├──>cap->vdev.fops = &vin_fops; // 文件接口的ioctl
								├──>cap->vdev.ioctl_ops = &vin_ioctl_ops; // ioctl的命令列表
								├──>video_register_device(cap->vdev)
								├──>video_set_drvdata()
								└──>vb2_queue_init()

2 具体到我使用的sensor mlx75027,具体的初始化流程

// drivers/media/platform/sunxi-vin/modules/sensor/mlx75027_mipi.c
module_init(init_sensor);
│	static struct i2c_driver sensor_driver = {
│			.probe = sensor_probe,
│			...
│	};
└──>init_sensor(void);
	└──>cci_dev_init_helper(&sensor_driver);
		└──>sensor_driver->probe()
			└──>sensor_probe()
				├──>sensor_init_controls(sensor_ctrl_ops);
				│	│  // 定义sensor对3A命令处理的逻辑,可覆盖父级别
				│	└──>v4l2_ctrl_new_std(V4L2_CID_XXX);			
				└──>cci_dev_probe_helper(sensor_ops); // 设置sensor_ops
					│  // 定义sensor对3A命令处理的逻辑,可覆盖父级别
					└──>struct v4l2_subdev_ops sensor_ops = {
						.core = &sensor_core_ops,
						...
						};
						└──>struct v4l2_subdev_core_ops sensor_core_ops = {
							.ioctl = sensor_ioctl, // 上层通过ioctl,最终会调用到这里,我实现的自定义命令,也会到这个处理
							...
							};

3.V4L2 cmd的类型

  1. 第一种是定义在 /include/uapi/linux/videodev2.h 中 VIDIOC_XXX
    这种命令的个数不超过 #define BASE_VIDIOC_PRIVATE 192 做为v4l2保留的命令,自定义的命令,不建议加在这个区间内.
    这些命令是v4l2操作的基本函数,包括设备的打开和关闭,流数据的请求等
/*
 *	I O C T L   C O D E S   F O R   V I D E O   D E V I C E S
 *
 */
#define VIDIOC_QUERYCAP		_IOR('V',  0, struct v4l2_capability)
#define VIDIOC_RESERVED		_IO('V',  1)
#define VIDIOC_ENUM_FMT     _IOWR('V',  2, struct v4l2_fmtdesc)
#define VIDIOC_G_FMT		_IOWR('V',  4, struct v4l2_format)
#define VIDIOC_S_FMT		_IOWR('V',  5, struct v4l2_format)
#define VIDIOC_REQBUFS		_IOWR('V',  8, struct v4l2_requestbuffers)
#define VIDIOC_QUERYBUF		_IOWR('V',  9, struct v4l2_buffer)
#define VIDIOC_G_FBUF		_IOR('V', 10, struct v4l2_framebuffer)
#define VIDIOC_S_FBUF		_IOW('V', 11, struct v4l2_framebuffer)
#define VIDIOC_OVERLAY		_IOW('V', 14, int)
#define VIDIOC_QBUF			_IOWR('V', 15, struct v4l2_buffer)
#define VIDIOC_EXPBUF		_IOWR('V', 16, struct v4l2_exportbuffer)
#define VIDIOC_DQBUF		_IOWR('V', 17, struct v4l2_buffer)
#define VIDIOC_STREAMON		_IOW('V', 18, int)
#define VIDIOC_STREAMOFF	_IOW('V', 19, int)
  1. 第二种是定义在 /include/uapi/linux/v4l2-controls.h 中 V4L2_CID_XXX
    因为是应用层使用,所以这里定义了地址,并按照一定的大小来区分不同的应用可以使用的范围.
#define V4L2_CTRL_CLASS_USER	0x00980000	/* Old-style 'user' controls */
#define V4L2_CID_BASE			(V4L2_CTRL_CLASS_USER | 0x900)
#define V4L2_CID_USER_BASE 		V4L2_CID_BASE
#define V4L2_CID_USER_CLASS 	(V4L2_CTRL_CLASS_USER | 1)
#define V4L2_CID_BRIGHTNESS		(V4L2_CID_BASE+0)
#define V4L2_CID_CONTRAST		(V4L2_CID_BASE+1)
  1. 第三种是定义在 /include/media/sunxi_camera_v2.h 里面给各个Camera使用的cmd.
    可以看到在私有的192之后,可以添加sensor的私有数据 VIDIOC_ISP_XXXVIDIOC_VIN_SENSOR_XXX .
#define BASE_VIDIOC_PRIVATE	192
#define VIDIOC_ISP_AE_STAT_REQ  _IOWR('V', BASE_VIDIOC_PRIVATE + 1, struct isp_stat_buf)
#define VIDIOC_ISP_HIST_STAT_REQ _IOWR('V', BASE_VIDIOC_PRIVATE + 2, struct isp_stat_buf)
...
#define VIDIOC_VIN_SENSOR_CFG_REQ _IOWR('V', BASE_VIDIOC_PRIVATE + 60, struct sensor_config)
#define VIDIOC_VIN_SENSOR_EXP_GAIN _IOWR('V', BASE_VIDIOC_PRIVATE + 61, struct sensor_exp_gain)
#define VIDIOC_VIN_SENSOR_SET_FPS  _IOWR('V', BASE_VIDIOC_PRIVATE + 62, struct sensor_fps)
...
// 我新增的命令放在了最后
struct msg_i2c {
	unsigned short addr;
	unsigned short value;
};
#define VIDIOC_VIN_SET_I2C_DATA  _IOWR('V', BASE_VIDIOC_PRIVATE + 76, struct msg_i2c)

4.不同类型命令的处理过程

1. VIDIOC_XXX 的处理

	int sel = 0;
	int mode = 5;
	enum v4l2_buf_type type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
	if (-1 == camera_init(sel, mode))
		return -1;
	if (-1 == camera_fmt_set(mode))
		return -1;
	if (-1 == req_frame_buffers())
		return -1;

	pixformat = TVD_PL_YUV420;
	disp_init(input_size.width, input_size.height, pixformat);
	if (-1 == ioctl(fd, VIDIOC_STREAMON, &type))
	{
		printf("VIDIOC_STREAMON failed\n");
		return -1;
	}

在 ioctl 之后,从kernel 到 driver 的流程如下

// 在 第1节已经知道了  vin_init_video() 会注册 cap->vdev.fops = &vin_fops; 
// 所以对vin设备的ioctl都会到这个函数
static struct v4l2_file_operations vin_fops = {
	.unlocked_ioctl = video_ioctl2,
}
static struct v4l2_ioctl_info v4l2_ioctls[] = {
	IOCTL_INFO_FNC(VIDIOC_STREAMON, v4l_streamon, v4l_print_buftype, INFO_FL_PRIO | INFO_FL_QUEUE),
	IOCTL_INFO_FNC(VIDIOC_STREAMOFF, v4l_streamoff, v4l_print_buftype, INFO_FL_PRIO | INFO_FL_QUEUE),
}

EXPORT_SYMBOL(video_ioctl2);
video_ioctl2()
└──>__video_do_ioctl()
	├──>v4l2_is_known_ioctl() // 如果是在已知cmd列表里面,就执行注册的func
	├──>struct v4l2_ioctl_info *info = &v4l2_ioctls[_IOC_NR(cmd)];
	└──>info->func(ops, file, fh, arg);
		└──>v4l_streamon()
			└──>ops->vidioc_streamon(file, fh, *(unsigned int *)arg);
				└──>static const struct v4l2_ioctl_ops vin_ioctl_ops = {
					.vidioc_streamon = vidioc_streamon,
					.vidioc_streamoff = vidioc_streamoff,
					}
					// /drivers/media/platform/sunxi-vin/vin-video/vin_video.c
					vidioc_streamon()
					└──>vb2_streamon()
						│ // /drivers/media/v4l2-core/videobuf2-core.c
						└──>vb2_core_streamon()
							├──>v4l_vb2q_enable_media_source()
							└──>vb2_start_streaming()
								│ // /drivers/media/platform/sunxi-vin/modules/sensor/mlx75027_mipi.c
								└──>struct v4l2_subdev_video_ops sensor_video_ops = {
									.s_stream = sensor_s_stream,
									}
									sensor_s_stream()
									└──>sensor_s_streamon(sd,1);
										└──>sensor_write(sd, 0x1001, 0x01); // 实际操作寄存器

2. V4L2_CID_XXX 的处理

// drivers/media/platform/sunxi-vin/modules/sensor/mlx75027_mipi.c
module_init(init_sensor);
│	static struct i2c_driver sensor_driver = {
│			.probe = sensor_probe,
│			...
│	};
└──>init_sensor(void);
	└──>cci_dev_init_helper(&sensor_driver);
		└──>sensor_driver->probe()
			└──>sensor_probe()
				├──>sensor_init_controls(sensor_video_ops);
				│	│  // 定义sensor对3A命令处理的逻辑,可覆盖父级别
				│	└──>v4l2_ctrl_new_std(V4L2_CID_XXX);			
				└──>cci_dev_probe_helper(sensor_ops); // 设置sensor_ops
					│  // 定义sensor对3A命令处理的逻辑,可覆盖父级别
					└──>v4l2_subdev_video_ops sensor_video_ops = {
							.s_parm = sensor_s_parm,
							.g_parm = sensor_g_parm,
						...
						};
						└──>struct v4l2_ioctl_ops soc_camera_ioctl_ops = {
							.vidioc_g_parm		 = soc_camera_g_parm,
							}
							└──>soc_camera_g_parm()
								└──>default_g_parm()
									└──>vidioc_g_parm()
										└──>v4l2_subdev_call(vinc->vid_cap.pipe.sd[VIN_IND_SENSOR], video, g_parm, parms);
											└──>in_ctrl_ops = {
												.g_volatile_ctrl = vin_g_volatile_ctrl,
												};
												└──>vin_g_volatile_ctrl()
													// /drivers/media/platform/sunxi-vin/modules/sensor/mlx75027_mipi.c
													v4l2_ctrl_ops sensor_ctrl_ops = {
														.g_volatile_ctrl = sensor_g_ctrl,
													};
													└──>int sensor_g_ctrl(struct v4l2_ctrl *ctrl)
															switch (ctrl->id)
															case V4L2_CID_EXPOSURE:
																sensor_g_exp(); // 实际的处理函数

3. 自定义cmd的处理

 // /include/media/sunxi_camera_v2.h   在头文件增加新的cmd和传入的参数定义
struct msg_i2c {
	unsigned short addr;
	unsigned short value;
};
#define VIDIOC_VIN_SET_I2C_DATA	_IOWR('V', BASE_VIDIOC_PRIVATE + 76, struct msg_i2c)

// /drivers/media/platform/sunxi-vin/vin-video/vin_video.c
struct v4l2_ioctl_ops vin_ioctl_ops = {
	.vidioc_default = vin_param_handler, // 走的是这个函数
└──>vin_param_handler()
	└──>switch (cmd) {
		case VIDIOC_VIN_SET_I2C_DATA:
		vidioc_set_i2c_zhang(struct file *file, struct v4l2_fh *fh, struct msg_i2c *i2c);
		└──>struct vin_core *vinc = video_drvdata(file);
			v4l2_subdev_call(vinc->vid_cap.pipe.sd[VIN_IND_SENSOR], core, ioctl, VIDIOC_VIN_SET_I2C_DATA, i2c);
			│// /drivers/media/platform/sunxi-vin/modules/sensor/mlx75027_mipi.c
			└──>struct v4l2_subdev_ops sensor_ops = {
				.core = &sensor_core_ops,
				};
				struct v4l2_subdev_core_ops sensor_core_ops = {
				.ioctl = sensor_ioctl,
				}
				└──>sensor_ioctl()
					└──>switch (cmd) {
						case VIDIOC_VIN_SET_I2C_DATA:
				        sensor_s_i2c_data(sd, (struct msg_i2c *)arg);
				        └──> sensor_write(sd, i2c->addr, i2c->value);

4. v4l2_subdev_call() 这个是实现自定义cmd的重点

/*
 * Call an ops of a v4l2_subdev, doing the right checks against
 * NULL pointers.
 *
 * Example: err = v4l2_subdev_call(sd, video, s_std, norm);
 */
#define v4l2_subdev_call(sd, o, f, args...)				\
	(!(sd) ? -ENODEV : (((sd)->ops->o && (sd)->ops->o->f) ?	\
		(sd)->ops->o->f((sd), ##args) : -ENOIOCTLCMD))
vidioc_set_i2c_zhang(struct file *file, struct v4l2_fh *fh, struct msg_i2c *i2c);		
└──>v4l2_subdev_call(vinc->vid_cap.pipe.sd[VIN_IND_SENSOR], core, ioctl, VIDIOC_VIN_SET_I2C_DATA, i2c);

struct v4l2_subdev_ops sensor_ops = {
	.core = &sensor_core_ops,
	};
struct v4l2_subdev_core_ops sensor_core_ops = {
	.ioctl = sensor_ioctl,
	}

v4l2_subdev_call(vinc->vid_cap.pipe.sd[VIN_IND_SENSOR], core, ioctl, VIDIOC_VIN_SET_I2C_DATA, i2c);
实际上等价于
v4l2_subdev_call(subdevice, sensor_ops , ioctl, VIDIOC_VIN_SET_I2C_DATA, struct msg_i2c);
翻译过来就是通过subdevice来调用 sensor_ops 的 ioctl 方法, 传入的参数是VIDIOC_VIN_SET_I2C_DATA, 方法的结构体是struct msg_i2c.

5.遇到的问题

1. is_isp_used 和 is_bayer_raw 的值会影响 V4L2_CID_XXX 的处理逻辑

static int vin_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
{
	struct vin_vid_cap *cap = container_of(ctrl->handler, struct vin_vid_cap, ctrl_handler);
	struct sensor_instance *inst = get_valid_sensor(cap->vinc);
	if (inst->is_isp_used && inst->is_bayer_raw) {
	// 一般而言,V4L2_CID_XXX 会走这个分支
	} else {
	}
// 	/drivers/media/platform/sunxi-vin/vin.c
static int __vin_handle_sensor_info(struct sensor_instance *inst)
{
	if (inst->cam_type == SENSOR_RAW) {
		inst->is_bayer_raw = 1;
		inst->is_isp_used = 1;
	} else if (inst->cam_type == SENSOR_YUV) {
		inst->is_bayer_raw = 0;
		inst->is_isp_used = 0;
	} else {
		inst->is_bayer_raw = 0;
		inst->is_isp_used = 0;
	}
	return 0;
}
// 从这个函数可以看到,只有sensor的type是SENSOR_RAW的时候,这2个属性才都是1.
// 所以就需要在初始化列表里面将sensor的属性配置好,如果是新增的sensor,这2个值都是0.
// /drivers/media/platform/sunxi-vfe/utility/sensor_info.c
struct sensor_item sensor_list_t[] = {
	/* name       i2c_addr      sensor type  sensor size   sensor max pclk */
	{	"ov2640",	0x60,		SENSOR_YUV,	 PIXEL_NUM_2M, CORE_CLK_RATE_FOR_2M},
	{	"ov5647_mipi",	0x6c,	SENSOR_RAW,	 PIXEL_NUM_5M, CORE_CLK_RATE_FOR_5M},
	}
// 可以把新增的sensor添加在这个结构体的后面.	

2. linux 内核宏container_of的意思

linux 内核宏container_of剖析
关于linux container_of用法
container_of用法及实现
linux 驱动程序中 container_of宏解析
详解Linux内核之位操作

struct struct *struct_p= container_of(struct_member_p, struct struct, struct_member);
//container_of 的作用是:已知 struct结构中某个成员struct_member的指针struct_member_p,就可以知道整个struct的指针struct_p

6. 全志提供了V4L2设备直接读写I2C的做法

我是在封装完毕API之后,才在全志的官网看到了这个方法…现在分享出来.
【FAQ440】Tina在线读写Sensor寄存器示例
发布日期 2021-10-30 15:47:18
平台 V536 V533 V833 V831 就是不知道T507是否也支持,实际看了下,sensor目录下的所有文件都是一致的.

【命令示例】
1)cd /sys/devices/gc2053_mipi(进入目标 sensor 节点目录)
2)echo 16 > addr_width; echo 8 > data_width(输入目标 sensor 寄存器地址/数据位宽,请查阅 datasheet 获取)
3)echo 0 > read_flag(read_flag:读写控制节点,使能为1表示后续操作为读动作,使能为0表示后续操作为写动作)
4)echo 30350021 > cci_client(“30350021”:0x3035【目标寄存器地址】,0x0021【将要写入的寄存器值】,在 read_flag = 1 情况下,写入值为无效状态)
5)cat read_value(打印上一步操作的结果:寄存器值)

【注意事项】
1)IIC 在系统和硬件上正常工作;
2)上述命令需要在 Camera 正常工作状态下执行才会有效,如 /sys/devices/sensor 节点是否存在,Camera 是否上电等等;
3)上述命令只在 tina-linux 系统有效,rtos 系统暂不支持;

标签:struct,ops,V4L2,vin,全志,I2C,sensor,v4l2
来源: https://blog.csdn.net/scarlettsp/article/details/121208936