Traditional camera accumulates photons during an exposure time window to generate a still image or a video in the form of image sequences

which lose the temporal process of the photons flow

leads to an irreconcilable dilemma between high-dynamic and high-speed imaging. Based on the fact that the pixels of the photoelectric sensor are independent

a new continuous photographing principle is proposed: each pixel converts the received photon flow into an electronic flow independently

continuously measures the photoelectric current and converts it into a digital flow

and represents continuously at the pixel level. Then the sequence array ranked according to the pixel layout is a continuous representation of the photons shotting on the sensing plane. The image of any moment can be obtained by intercepting the state of the sequence array

so as to realize continuous imaging.

Furthermore

the spiking continuous photographing principle that modulates the photon flow into a spike sequence is proposed: for each pixel

accumulate charge from the reset state

generate a spike as a flag once the specified threshold is reached

reset and repeat. The duration that a spike takes to be fired is called its spiking width

which is inversely proportional to the light intensity during this period. Based on this

the light intensity during this period can be estimated. The spikes sequence with natural temporal order is a digital representation of the photon flow process. The array of the spike sequences according to the spatial layout of pixels is called “viform”

which contains rich spatial and temporal information of the light process. An image at any moment can be calculated from viform

thereby ultrahigh-speed

high-dynamic and non-blurred continuous imaging is achieved

and the dilemma between high-dynamic and high-speed imaging of exposure imaging is solved completely.

The unique parameter of the spiking continuous photographing principle is the accumulation threshold Q

which corresponds to the number of photons/electrons required to fire a spike. The unique variable is the spike accumulation time τ

which is the easiest to measure accurately and can be concisely represented by natural temporal order. The light intensity at any moment is Q/τ

which is determined by the spike width τ

which range is (0

∞). Thus

theoretically

arbitrary light intensity could by be represented

and lead to infinite dynamic range imaging. In the physical implementation

the strongest light could be represented depending on the shortest readout time τ̂ of the circuit

and the weakest light is the dark current

which can be accumulated as a spike within duration τ̂. τ̂/τ̌makes up the imaging dynamic range. With conventional photoelectric devices and circuits

ultrahigh-dynamic imaging with 160 dB or even 180 dB can be achieved. When the imaging time sensitivity is required to be less than τ̂

ultrahigh-dynamic imaging can be achieved by sub-threshold quantization of the accumulated voltage.

Photon and photoelectron flow are discrete statistical process that follows a Poisson distribution and is often a “segmental linear” process light intensity changes suddenly while the intensity is stable between successive mutations. A method so-called first spike encoding is proposed: only the moment of the first spike firing and its spike width are output when the intensity changes

and no spike is output thereafter to indicate repetition. This is the optimal lossless compression method of the spike sequence

where the dynamic range and time sensitivity can be significantly improved by improving the time measurement accuracy with almost no increase in representation data volume.

Using mature CMOS photonic devices and standard processes

two spike continuous photographing chips and spike cameras with spatial resolutions of 0.1 million and 1 million pixels have been developed

respectively. The chips use 40 000 Hz synchronous spike output

with a minimum spike width of 25 μs. Practical tests have verified the feasibility of the spike continuous photographing principle and its ability to achieve ultrahigh-speed

high-dynamic

and blur-free imaging.

Viform

the spike continuous photographing representation and the replacement of image and video

captures the temporal and spatial information of the photon flow

will fundamentally reshapes computer vision and visual information processing technology and industries. An open source algorithm framework

SpikeCV

achieves high-speed target detection

tracking

and recognition system that is faster than the human eye by thousands of times

with low computational complexity. As the symmetrical process of spike continuous photographing

spike continuous displaying modulates the ultrahigh-speed spike sequence into extremely high-speed photon flow

enabling high-speed display similar to natural light through glass

solving the motion blur and visual fatigue dizziness caused by low frame rate in traditional display systems. In addition

the combination of continuous photographing and continuous displaying can achieve a single-way transparent glass-like display and ultrahigh-speed light communication without medium.