Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable multicodec sound cards #6595

Open
wants to merge 753 commits into
base: rpi-6.12.y
Choose a base branch
from

Conversation

gordoste
Copy link

@gordoste gordoste commented Jan 9, 2025

This patch prepares to remove the custom ezsound patches that were introduced in the 6.12 tree.

As a reminder, devices with:

  • A single DAI link with multiple codecs
  • Clocks produced by one of the codecs
  • Other codec(s) and CPU are clock consumers

could not be configured properly under ASoC, and the patchset fixed the problem by having the codec driver override the configuration provided by SoC.

A patch series ("ext_fmt") has been developed and tested (email thread) to address the ASoC limitation, and should be introduced to the 6.13 mainline before too long. However, one of the changes made to the rpi tree as part of the usual soundcard support patch explicitly disables support for multiple codecs. This patch reverts that change.

I compiled kernels from your 6.13.y tree with and without this patch.

6.13.0-rc6 with ext_fmt patchset hangs on arecord (expected, since it has no clock)
6.13.0-rc6 with ext_fmt patchset plus this patch works perfectly (without any of the prior pcm3168a customisations).

I inserted debugging messages which show what is happening:

[   28.518387]  ezsound I2S PCM: DAI #0 - NOT tweaking dai_fmt
[   28.518390]  ezsound I2S PCM: DAI #1 - tweaking dai_fmt

You might wonder why the current ezsound patches that you accepted previously don't have this problem. It's because it sets up two separate DAI links, one for playback and one for capture, which both use the same CPU. This one is a single DAI link with 1 CPU and 2 codecs. As a result, the ALSA card has a single device used for both playback and capture, which is better IMHO.

Obviously, this is not urgent, but I thought I'd float it in case you wanted to make this change before the upcoming move to 6.12.

EDIT: Just realized I forgot Signed-Off-By. I can fix that of course, I was focussed on trying to explain this clearly.

pelwell and others added 30 commits January 2, 2025 10:45
Although later functions can handle a NULL fwnode, fwnode can't handle
being passed a NULL pointer.

See: raspberrypi#6305

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
The check if the oscillator stop bit is set was reading from Control_1
register instead of the Seconds register.
This caused the Seconds register to be incorrectly changed if bit 7 of
Control_1 happens to be set.

Signed-off-by: Axel Hammarberg <axel.hammarberg@gmail.com>
Add YAML device tree binding for the Sony IMX500 CMOS image sensor /
CNN inference engine.  Also, add a MAINTAINERS entry.

Signed-off-by: Richard Oliver <richard.oliver@raspberrypi.com>
The Sony IMX500 is a stacked 1/2.3-inch CMOS digital image sensor and
inbuilt AI processor with an active array CNN (Convolutional Neural
Network) inference engine.  The native sensor size is 4056H x 3040V, and
the module also contains an in-built ISP for the CNN. The module is
programmable through an I2C interface with firmware and neural network
uploads being made over SPI. This driver supports imaging only.

Signed-off-by: Richard Oliver <richard.oliver@raspberrypi.com>

media: i2c: imx500: Inbuilt AI processor support

Add support for the IMX500's inbuilt AI processor. The IMX500 program
loader, AI processor firmware, DNN weights are accessed via the kernel's
firmware interface on 'open' and are transferred to the IMX500 over SPI.

Signed-off-by: Richard Oliver <richard.oliver@raspberrypi.com>

media: i2c: imx500: Enable LED during SPI transfers

The Raspberry Pi 'AI Camera' is equipped with an LED. Enable this LED
during SPI transfers to indicate to the end-user that progress is being
made during large tramsfers.

Signed-off-by: Richard Oliver <richard.oliver@raspberrypi.com>

drivers: media: imx500: Fixes for vblank control

Reduce the default/max framerate of the 2x2 binned mode to 30fps.
The current limit of 50fps can cause the sensor to produce corrupt
frames and cause missing framing events.

Also fixup the vblank control min/max/default/step paramters when
setting up.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>

drivers: media: imx500: Simplify the vblank control init

Set the VBLANK control minimum and default values to IMX500_VBLANK_MIN
unconditionally everywhere.

Remove the mode specific framerate_default parameter, it is now unused.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add EXPORT_SYMBOL_GPL() for find_cpio_data() so that loadable modules
may also parse uncompressed cpio.

Signed-off-by: Richard Oliver <richard.oliver@raspberrypi.com>
The driver did expose V4L2_CID_HBLANK, but as a READ_ONLY control.

The sensor only uses the HTS register to control the line length,
so convert this control to read/write, with the appropriate ranges.
Adopt the old fixed values as the minimum values permitted in each
mode to avoid issues of it not streaming.

This should allow exposure times up to ~3 seconds (up from ~1sec).

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
ili9881c_enable is always returning 0.

ili9881c_disable was returning the error code from
mipi_dsi_dcs_set_display_off.
If non-zero, the drm_panel framework will leave the panel marked as
enabled, and not run the enable hook next time around. That isn't
helpful, particularly as we're expecting unprepare to disable
resets and regulators.

Change ili9881c_disable to match enable in always returning 0.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
The driver is typically asking for LP commands, but then tries
to send set_display_[on|off] from enable/disable when the host
will be in HS mode.
It also sends shutdown commands just before it asserts reset and
disables the regulator, which is rather redundant.

Add an option to configure these two choices from the panel_desc.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
This correction is calibrated to approx 5000K.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
The i.MX8MP makes calls on it's source device to determine
the link-frequency that should be configured on the CSI2 receiver.

When the source is behind a video mux, we need to pass this call through
to the connected device.

Map the control handler of the source device to the video-mux,
essentially proxying all controls on the mux to the device which has
it's link enabled.

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
The current order of locking between the driver mutex and the v4l2
subdev state lock causes a circuluar locking dependency when trying to
set up a link. Fix this.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
To make comparisons of the mode registers easier, put the registers
for the binned and VGA modes in the same order as the others.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
There are many registers in common between all the modes.
Pull those out into one common table.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
…ed modes

In order to simplify the driver slightly, use the same PLL
configuration, and hence pixel rate and link frequency (to be
added) for the full, 1080p, and binned modes.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
The link frequency can vary between modes, so add it as a
control.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
system_heap.max_order=<uint>

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Add numa_policy kernel argument to allow overriding the kernel's default
NUMA policy at boot time.

Syntax identical to what tmpfs accepts as it's mpol argument is accepted.

Some examples:

 numa_policy=interleave
 numa_policy=interleave=skip-interleave
 numa_policy=bind:0-3,5,7,9-15
 numa_policy=bind=static:1-2

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
... Make sure CMA zones do not straddle the emulated NUMA nodes ...

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Clients such as libcamera do not change the LS table dmabuf on every
frame. In such cases instead of mapping/remapping the same dmabuf on
every frame to send to the firmware, cache the dmabuf once and only
update and remap if the dmabuf has been changed by the userland client.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
As a workaround (and possibly a fix) for CPU spins observed on BCM2837,
use ptep_clear_flush_young instead of ptep_test_and_clear_young inside
lru_gen_look_around in order to expose PTE changes to the MMU. Note that
on architectures that don't require an explicit flush,
ptep_clear_flush_young just calls ptep_test_and_clear_young.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
A user has reported that a card of this model from late 2021 doesn't
work, so extend the date range and make it match on all card sizes.

Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
Add two quirk properties that control whether or not the controller
issues many more handshakes to FS/HS Async endpoints in a single
(micro)frame. Enabling these can significantly increase throughput for
endpoints that frequently respond with NAKs.

Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
If a device frequently NAKs, it can exhaust the scheduled handshakes in
a frame. It will then not get polled by the controller until the next
frame interval. This is most noticeable on FS devices as the controller
schedules a small set of transactions only once per full-speed frame.

Setting the ENH_PER_NAK_FS/LS bits in the GUCTL1 register increases the
number of transactions that can be scheduled to Async (Control/Bulk)
endpoints in the respective frame time. In the FS case, this only
applies to FS devices directly connected to root ports.

Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
For platforms that have xHCI controllers attached over PCIe, and
non-coherent routes to main memory, a theoretical race exists between
posting new TRBs to a ring, and writing to the doorbell register.

In a contended system, write traffic from the CPU may be stalled before
the memory controller, whereas the CPU to Endpoint route is separate
and not likely to be contended. Similarly, the DMA route from the
endpoint to main memory may be separate and uncontended.

Therefore the xHCI can receive a doorbell write and find a stale view
of a transfer ring. In cases where only a single TRB is ping-ponged at
a time, this can cause the endpoint to not get polled at all.

Adding a readl() before the write forces a round-trip transaction
across PCIe, definitively serialising the CPU along the PCI
producer-consumer ordering rules.

Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
The DHT11 datasheet is pretty cryptic, but it does suggest that after
each integer value (humidity and temperature) there are "decimal"
values. Validate these as integers in the range 0-9 and treat them as
tenths of a unit.

Link: raspberrypi#6220

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
…ctors

The non-desktop property "Indicates the output should be ignored for
purposes of displaying a standard desktop environment or console."

That sounds like it should be true for all writeback and virtual
connectors as you shouldn't render a desktop to them, so set it
by default.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
The limit of 32 planes per DRM device is dictated by the use
of planes_mask returning a u32.

Change to a u64 such that 64 planes can be supported by a device.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Some hardware will implement transpose as a rotation operation,
which when combined with X and Y reflect can result in a rotation,
but is a discrete operation in its own right.

Add an option for transpose only.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Some connectors, particularly writeback, can implement flip
or transpose operations as writing back to memory.

Add a connector rotation property to control this.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
For devices where transfer lengths are not known upfront, there is a
danger when the destination is wider than the source that partial words
can be lost at the end of a transfer. Ideally the controller would be
able to flush the residue, but it can't - it's not even possible to tell
that there is any.

Instead, allow the client driver to avoid the problem by setting a
smaller width.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
herbingk and others added 23 commits January 2, 2025 10:46
Add and update files for pwm-gpio-fan overlay
Signed-off-by: Kai-Uwe Herbing <155751635+herbingk@users.noreply.github.com>
on 32-bit kernels, folio_seek_hole_data() was inadvertently truncating a
64-bit value to 32 bits, leading to a possible infinite loop when writing
to an xfs filesystem.

Signed-off-by: Marco Nelissen <marco.nelissen@gmail.com>
Commit ec75fd952b0b5cdab7b606cdacba237c57c1fdda upstream.

The imx327 only supports up to 29.4dB of analogue gain, vs
the imx290 going up to 30dB. Both are in 0.3dB steps.

As we now have model specific config, fix this mismatch,
and delete the comment referencing it.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Alexander Stein <alexander.stein@ew.tq-group.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Commit f2055c1d62d6dfd25a31d1d1923883f21305aea5 upstream.

Reviewing the datasheets, register 0x3011 is meant to be 0x02 on imx327
and 0x00 on imx290.

Move it out of the common registers, and set it appropriately in the
sensor specific sections. (Included for imx290 to be explicit, rather
than relying on the default value).

Fixes: 2d41947 ("media: i2c: imx290: Add support for imx327 variant")
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Commit e4faac99d5bb4b6c80f2495c40fcd71a67c40b27 upstream.

IMX462 is the successor to IMX290, which is supportable by
the existing IMX290 driver via a new compatible string.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Commit c699b6c7c857baba1375a1ed090bf71f695e2971 upstream.

IMX462 is the successor to IMX290, and wants very minor
changes to the register setup.

Add the relevant configuration to support it.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The sensor has Low Conversion Gain (HCG) and High Conversion Gain (HCG)
modes, with the supposedly the HCG mode having better noise performance
at high gains.

As this parameter changes the gain range of the sensor, it isn't
possible to make this an automatic property, and there is no
suitable V4L2 control to set it, so just add it as a module parameter.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Now that imx462 has a separate compatible string, make use of it.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
In commit b478e16 ("PCI/ASPM: Consolidate link state defines")
PCIE_LINK_STATE_L1 and PCIE_LINK_STATE_L0s grew some bits for more
granular control of ASPM.

This broke the aspm-no-l0s override, instead disabling link ASPM
completely if this DT property was specified.

Just open-code the link capability register field bits.

Fixes: caab002 ("PCI: brcmstb: Disable L0s component of ASPM if requested")
Fixes: 0693b42 ("PCI: brcmstb: Split post-link up initialization to brcm_pcie_start_link()")
Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>
The ARM PL011 UART instances in BCM2711 are r1p5 spec, which means they
have 32-entry FIFOs. The correct periphid value for this is 0x00341011.
Thanks to N Buchwitz for pointing this out.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
The ARM PL011 UART instances in BCM2712 are r1p5 spec, which means they
have 32-entry FIFOs. The correct periphid value for this is 0x00341011.
Thanks to N Buchwitz for pointing this out.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
The ARM PL011 UART instances in RP1 are r1p5 spec, which means they
have 32-entry FIFOs. The correct periphid value for this is 0x00341011.
Thanks to N Buchwitz for pointing this out.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
After commit 5fd3e24 ("media: v4l2-subdev: Support hybrid links
in v4l2_subdev_link_validate()") link_validate is called on V4L2
OUTPUT devices such as the CFE cfg buffers input.
The CFE link_validate function was assuming it was always the
sink of a link, which goes wrong on that port and does an invalid
dereference.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
This also requires that the ranges for the exposure control
are updated.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
The control of HMAX documented in the datasheet is consistent
with being in terms of a scaled INCK, being always 72MHz or
74.25MHz. It is NOT link frequency dependent, but the minimum
value for HMAX is dictated by the link frequency.

If PIXEL_RATE is defined as being 12 times the 72 or 74.25MHz,
and all values are scaled down again when writing HMAX, then
the numbers all work out regardless of INCK or link frequency.
Retain an hmax_min (set to the same value as the previous fixed
hmax register value) to set as the default value to avoid changing
the behaviour for existing users.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
The link frequencies are equally valid in 2 or 4 lane modes, but
they change the hmax_min value for the mode as the MIPI block
has to have sufficient time to send the pixel data for each line.

Remove the association with number of lanes, and add hmax_min
configuration for both lane options.

Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Designing a PCM512X-based soundcard with no external SCLK is a valid
choice supported by the driver. Don't alarm users with messages that
say "No SCLK, using BCLK: -2" - reclassify them as debug information.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Controls which only exist when snd_soc_register_card returns can't be
modified before then. Move the setting of volume limits to just before
the end of the probe function.

Link: raspberrypi#6527

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
The codec control Digital Playback Volume is one of the controls deleted
by the allo-piano-dac-plus driver. It is effectively replaced by the
soundcard controls Master Playback Volume and Subwoofer Playback Volume.

Delete the code that sets the volume limit on those codec controls - the
limits on the soundcard volume controls are sufficient.

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
This patch (which is not in upstream) prevents the I2S producing
clocks for multiple consumers.
@pelwell
Copy link
Contributor

pelwell commented Jan 10, 2025

This will have to be rebased before merging, but hold fire for now until I work out why the change was originally added.

@pelwell
Copy link
Contributor

pelwell commented Jan 10, 2025

Well, here's the where: #3337

@pelwell
Copy link
Contributor

pelwell commented Jan 10, 2025

It might speed things up if you could share any thoughts you have about the patch you are reverting, and how the use case outlined can be preserved.

@gordoste
Copy link
Author

Thanks for digging up why this was originally changed. It looks the Justboom folks had the exact same challenge that I had due to the lack of ASoC support for multicodec setups with a codec as master. The "ext_fmt" patchset I mentioned previously will remove this constraint and allow creation of a DT overlay to support the JustBoom Digi + JustBoom DAC scenario using audio_graph_card2 driver. But that patchset won't be in until 6.13.
I can see 3 options:

  1. Undo Add Support for simultaneous use of JustBoom DAC and JustBoom Digi based Audio boards #3337 in 6.12.y. Users of the JustBoom Digi + JustBoom DAC will need to stay on 6.6 forever. PR Add support for ezsound 6x8 soundcard #6564 will stay in 6.12 but be removed after that.
  2. Undo Add Support for simultaneous use of JustBoom DAC and JustBoom Digi based Audio boards #3337 in 6.13.y. Users of the JustBoom Digi + JustBoom DAC will need to stay on 6.12 unless the JustBoom guys add support to 6.13. PR Add support for ezsound 6x8 soundcard #6564 will stay in 6.12 but be removed after that.
  3. Keep PR Add support for ezsound 6x8 soundcard #6564 and PR Add Support for simultaneous use of JustBoom DAC and JustBoom Digi based Audio boards #3337 permanently.
    Note that a lot of the things on JustBoom's site are out of stock (including the JustBoom DAC HAT). It may be unlikely any further development will occur, particularly for old hats.

I suspect the best option is the 2nd one.... if you agree, then you can close this and once the "ext_fmt" patchset actually gets accepted into upstream, I will resubmit it.

@gordoste
Copy link
Author

FYI, the "ext_fmt" patch set was accepted and should get into the next merge window.

@pelwell
Copy link
Contributor

pelwell commented Jan 14, 2025

I'm happy to accept backports of upstream commits once merged to linux-next (at least). The cleanest solution appears to be to revert both patches and replace them with suitable use of the ext_fmt feature. I don't know if @shawaj still has an interest in the maintenance of the JustBoom drivers, but either way it would be appreciated if you created a best-effort update of any JustBoom software support that needs to change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.