[Tizen General] Tizen Audio Stack

Counihan, Tom tom.counihan at intel.com
Tue Sep 17 18:06:00 GMT 2013



> -----Original Message-----
> From: general-bounces at lists.tizen.org [mailto:general-bounces at lists.tizen.org]
> On Behalf Of Patrick Shirkey
> Sent: Thursday, September 12, 2013 8:29 AM
> To: Tizen List
> Subject: Re: [Tizen General] Tizen Audio Stack
> 
> 
> On Thu, September 12, 2013 10:03 pm, Uimonen, Jaska wrote:
> > Hello,
> >
> > Sorry for coming so late into this discussion...
> > and maybe starting to dig into the can of worms.
> >
> >>I didn't want to push that option as I fear it could open a can of
> >>worms especially as there are a couple of Intel guys in the PA team
> >>who are  paid to work on PA and Tizen integration and I don't want
> >>them to think they are under attack.
> >
> > Don't know If I'm one of the guys referred by this, but there's no
> > guys who are paid for working especially with PA. We are working on
> > solution that will do the use cases the industry wants. Valid use case
> > is not "I want Jack", but "I want Jack, because PA can't do this and
> > that".  We are very willing to discuss whatever solution makes sense,
> > so don't mind the politics :)
> >
> >>To really cut to the chase:
> >>
> >>- JACK is aimed at enabling realtime multimedia production in a
> >>modular environment
> >>- PA is aimed at audio management in a consumer environment where
> >>realtime is not the core concern
> >
> > So which are you thinking a car or phone is more: consumer environment
> > or multimedia production system?
> >
> 
> Typically they are both consumer environments and PA is the obvious choice for
> managing the audio system.
> 
> > I have seen mobile phones doing call audio streaming in user space
> > with PA.
> > I'm just saying phone audio is not that real time critical as audio
> > recording in professional studio. And I don't see many hard real time
> > media production related activities happening in car or mobile phone.
> > I could be wrong and every audio related stuff happening around tizen
> > is valuable :)
> >
> 
> I can see a use case for professional audio in both environments but I don't
> think it will be the default for most users.

In automotive IVI systems we have strict latency constrains to meet VDA specification for hands-free phone communication in the car. This leads into signal latency optimized designs, ensuring phone uplink and phone downlink signals will be routed and processed within 50 msec within the device (= mobile phone AND sound system in the car). This is stripped down to 20...30 ms left for audio processing (incl. async. sampling rate conversion) and routing after the BT transmission (between the mobile phone and the sound system in the car) and way down to the loudspeaker resp. the microphone. If you miss these conditions, hands-free phone calls out of the car will be a mess.
So while a dual combination of Jack and Pulse may be technically achievable, we need to be cognisant that every additional component causing signal latency between connected handset and the automotive speaker system may make it challenging to achieve such non-functional requirements.

> 
> 
> >> I am happy to put effort into making JACK a viable solution for the
> >> Tizen platform. IMO Adding support for BlueZ would be a very useful
> >> addition to JACK especially if it makes JACK a viable option for
> >> inclusion in Tizen.
> >
> > If people want Jack couldn't we just use jack and pulseaudio little
> > like they are used in desktops? So you could download Jack if you want
> > to and from pulseaudio we could treat jack as pulseaudio sink. I mean
> > from pulseaudio point of view Jack would look to us like external avb
> > amplifier with dsp or something similar.
> > I understand this would cause extra buffering and process hops, but
> > still could be maybe manageable. Also we should not need to convert
> > existing apps etc.
> >
> 
> I agree that a combination of PA and JACK solves many if not all of these
> problems.
> 
> > What we have inside pulseaudio is quite a lot of policy management and
> > I see this moving inside jack very problematic. On the other hand we
> > should soon handle before mentioned external "intelligent" amplifiers
> > and from policy point of view Jack would be just another external
> > processing unit.  Anyway, I find it very difficult to compare PA and
> > Jack and say that other one is absolutely better than the other. It is
> > so much depending what you are doing. At the moment we haven't seen
> > any crucial requirements to move to jack so we are using PA.
> >
> 
> I agree with you that PA is best suited to handling the default use case
> for the Tizen Audio Stack and your focus is in the right direction.
> 
> 
> > br,
> > Jaska Uimonen
> >
> >
> > btw if you guys are attending linuxcon or als in Edinburg, I'm also there
> > from wed onward with my colleagues so we could have meeting to
> > discuss the architectural issues.
> >
> 
> I'm not attending but could schedule a call if you want to discuss things
> directly.
> 
> Part of this conversation is attempting to identify the issues that could
> make it difficult to allow JACK to run on Tizen. To recap the discussion
> there are three issues identified so far:
> 
> 1: Automated switching between PA and  JACK with jack-sink
> 2: Handling Phone calls with PA/JACK while other resource intensive audio
> apps are running
> 3: Communicating with bluetooth devices while JACK is running
> 
> The issues surrounding each item are mainly about stability/integration.
> So far there has not been much testing done to isolate any unknown bugs
> and fine tune the automated logic for a mobile experience where all of the
> above is required. It is not that difficult to emulate these things in a
> desktop env for testing purposes so I don't see a major issue from a
> development pov.
> 
> However the possibility of running JACK directly on Tizen devices will be
> much more difficult to accomplish if PA (and GStreamer) don't have JACK
> support enabled ootb and have to be recompiled or (worse)
> replaced/disabled in order to run JACK on Tizen. I'm not sure what the
> current status is because I haven't got that far with testing audio on
> Tizen but it seems like jack-sink is not enabled by default at the moment?
> Is it just a case of making some config tweaks to enable jack-sink or does
> PA need to be recompiled?
> 
> I would like to see PA and JACK coexist in peaceful harmony, complementing
> each other by doing the things they do best. If that is not viable for
> some as yet unknown reason then I would like to make any suggested
> additions to JACK so that it can run on Tizen without being disruptive for
> normal use. My general position is that JACK is required for professional
> audio/multimedia and I would like it to be a valued member of the Tizen
> Audio Stack.
> 
> 
> 
> 
> 
> >
> >
> >
> >
> > ________________________________________
> > From: general-bounces at lists.tizen.org [general-bounces at lists.tizen.org] on
> > behalf of Patrick Shirkey [pshirkey at boosthardware.com]
> > Sent: Thursday, September 12, 2013 1:21 PM
> > To: Tizen List
> > Subject: Re: [Tizen General] Tizen Audio Stack
> >
> > On Thu, September 12, 2013 6:13 pm, Counihan, Tom wrote:
> >>
> >>
> >>> -----Original Message-----
> >>> From: general-bounces at lists.tizen.org
> >>> [mailto:general-bounces at lists.tizen.org]
> >>> On Behalf Of Patrick Shirkey
> >>> Sent: Friday, September 06, 2013 5:18 PM
> >>> To: Tizen List
> >>> Subject: Re: [Tizen General] Tizen Audio Stack
> >>>
> >>>
> >>> On Fri, September 6, 2013 7:07 pm, Counihan, Tom wrote:
> >>> > Hi Patrick,
> >>> >
> >>> >> On Thu, September 5, 2013 8:30 pm, Clark, Joel wrote:
> >>> >> > I know of other Tizen user that would like to see JACK in Tizen,
> >>> >> > but nobody has volunteered to contribute and maintain a port of
> >>> >> > JACK to the Tizen audio subsystem.  Any port of JACK would have to
> >>> >> > complement the existing Tizen audio components and not break
> >>> >> > working Tizen uses and applications and APIs.
> >>> >> >
> >>> >>
> >>> >> The JACK Developers will work within the bounds of specified
> >>> >> requirements for the Tizen Audio stack.
> >>> >>
> >>> >> The existing desktop logic is that when JACK is running PA hands
> >>> over
> >>> >> control of the ALSA interface and then optionally attaches to JACK
> >>> >> for I/O. Both Pulse and GStreamer can connect directly to JACK so
> >>> >> allowing consumer apps to continue to work while JACK is running is
> >>> >> not an issue in that regard.
> >>> >>
> >>> >> IIUC there has not been any specific testing done around the issue
> >>> of
> >>> >> handling phone calls while JACK is running on the desktop but that
> >>> >> issue has been solved for the iOS port. One option is to allow
> >>> system
> >>> >> level commands to interrupt/pause/disable JACK and take control of
> >>> >> the audio device and the other is to *optionally* allow the user to
> >>> >> ignore phones calls while JACK is running.
> >>> >> For professionals the latter is probably more appropriate as it's
> >>> >> unlikely a DJ would want to have a phone call interrupt them in the
> >>> >> middle of a set but for people who are just playing with their toys
> >>> >> the former is a completely reasonable way to handle things.
> >>> >
> >>> > Do you know if any work has been done by the JACK community to
> >>> > integrate Bluez? Specifically for HFP and A2DP/ARVCP use cases?
> >>> >
> >>>
> >>> AFAIK no one has publicly integrated JACK directly with the bluez API.
> >>> I
> >>> think
> >>> that is mostly because PA has pretty good support already. I haven't
> >>> come
> >>> across anyone using a bluetooth device with JACK yet but there is no
> >>> reason it
> >>> cannot or should not be supported. Artists are always looking for new
> >>> ways to
> >>> express themselves via technology.
> >>>
> >>> Currently there is support for ALSA, OSS and FFADO at the device layer.
> >>> Multichannel duplex USB devices can be accessed via ALSA or OSS and
> >>> IIUC
> >>> they have very similar performance limitations to bluetooth.
> >>>
> >>> Some people have had success working with wireless networked devices
> >>> with
> >>> some limitations on the i/o channel count and cabled networked audio is
> >>> definitely not a problem even distributed across the globe with the
> >>> right
> >>> infrastructure and bandwidth.
> >>>
> >>> Would you like JACK to send audio data directly to the bluetooth
> >>> headset
> >>> and
> >>> for the audio system to bypass PA while JACK it is running? If so would
> >>> it be a
> >>> requirement for JACK to send audio to both ALSA and bluez devices at
> >>> the
> >>> same
> >>> time?
> >>
> >> Actually, I come from the perspective of automotive, where the roles you
> >> are probably envisioning are reversed.
> >> And that said, I suspect also the audio quality demands, including
> >> latency
> >> demands, may be higher that you probably perceive above. Particularly in
> >> high-end vehicles, where the audio experience is presented as a key
> >> feature .. High-Def HFP, Blueray, ...
> >>
> >
> > There is at least one luxury car company that is investing time into JACK
> > support for their multimedia solutions.
> >
> > http://magnetimarelli.com/
> >
> > I'm not sure what Tesla have settled on but they most definitely use Linux
> > as their multimedia OS.
> >
> >
> >>> Otherwise we might be able to split the load between PA and JACK. I'm
> >>> not sure how well JACK would respond to sitting on top of PA but that
> >>> is
> >>> also
> >>> another potential option. It just hasn't been addressed on the desktop
> >>> as things
> >>> went the other way with PA letting JACK take over the device layer.
> >>
> >> This is one of a number of permutations you mention. One additional
> >> alternative not mentioned to date of course is a pure Jack solution.
> >> The reason I mention this is, as above highlights to me, it looks
> >> complicated interleaving the two solutions.
> >> With complication comes additional risk in terms of maintaining quality,
> >> stability and integrity. Also I'm guessing from a product perspective,
> >> the
> >> more code the higher the maintenance cost.
> >> I've no idea what the effort would be to offer an alternative JACK
> >> solution that could seamlessly slip into the Tizen architecture. But I
> >> suspect for those that control the architecture that would be a
> >> pre-requisite (I'm not one of those folks).
> >>
> >
> > I didn't want to push that option as I fear it could open a can of worms
> > especially as there are a couple of Intel guys in the PA team who are paid
> > to work on PA and Tizen integration and I don't want them to think they
> > are under attack.
> >
> > After many years of discussions between PA and JACK devs it is unlikely
> > that the two projects will be consolidated into a single solution because
> > they have broadly different goals and the management of the development
> > process explicitly requires different and conflicting priorities.
> >
> > The crucial difference between JACK and PA is that JACK is designed for
> > low latency professional audio where hardware/power/bandwidth/etc... is
> > not a physical limitation or even really a concern whereas PA is designed
> > for desktop audio management for the consumer experience with a lot of
> > effort spent on low power mobile where resources can be severely
> > restricted.  PA uses almost the same ringbuffer technique as JACK for
> > managing the low latency part of the problem. IIRC Lennart Poettering (PA)
> > came up with a slightly different implementation but he discussed it in
> > detail with Paul Davis and Stephane Letz (JACK) at the time.
> >
> > Based on those points of difference JACK enables many to many audio
> > routing, midi, networking and session management. PA enables the i/o
> > device to be shared between many applications and across a network or
> > bluetooth device. PA is not explicitly designed for audio to be shared
> > between apps. The main design goal is to enable normal users to be able to
> > hear audio i/o from their browser, skype and VLC at the same time
> > seamlessly.
> >
> > To really cut to the chase:
> >
> > - JACK is aimed at enabling realtime multimedia production in a modular
> > environment
> > - PA is aimed at audio management in a consumer environment where
> realtime
> > is not the core concern
> >
> >
> >
> >>
> >>>
> >>> I can see AVRCP being a useful addition to the control protocols. It
> >>> would
> >>> complement midi and osc and be very handy for transport control.
> >>> Enabling
> >>> Handsfree transport control would be a very powerful feature for
> >>> drummers or
> >>> other musicians/performers/artists who cannot easily physically touch
> >>> the
> >>> device. In the past that issue was dealt with by using a gate filter as
> >>> the trigger
> >>> to start/stop the transport but voice control would be very useful too.
> >>>
> >>> Can you see any other issues that I can address with the JACK/PA devs?
> >>
> >> Can you share your desired goals for Tizen (IVI)?
> >>
> >
> > My goal is to enable JACK to run on Tizen (IVI) without having to jump
> > through hoops. The ideal situation would be official support for JACK as
> > the professional audio solution alongside PA as the default audio
> > solution. It would cause me pain if for example OpenSL or "yet another
> > partial solution based on JACK" was given priority (and funding).
> >
> > I personally would like to be able to use JACK on Tizen for realtime
> > networked 3d multimedia production. It would enable cameras, actors,
> > models/set design, sound, all rendering in realtime with contributors from
> > across the globe.  In combination with other open source multimedia tools
> > that is a very powerful and cost effective distributed production and live
> > performance solution.
> >
> > I am happy to put effort into making JACK a viable solution for the Tizen
> > platform. IMO Adding support for BlueZ would be a very useful addition to
> > JACK especially if it makes JACK a viable option for inclusion in Tizen.
> >
> >
> >
> > --
> > Patrick Shirkey
> > Boost Hardware Ltd
> > _______________________________________________
> > General mailing list
> > General at lists.tizen.org
> > https://lists.tizen.org/listinfo/general
> > ---------------------------------------------------------------------
> > Intel Finland Oy
> > Registered Address: PL 281, 00181 Helsinki
> > Business Identity Code: 0357606 - 4
> > Domiciled in Helsinki
> >
> > This e-mail and any attachments may contain confidential material for
> > the sole use of the intended recipient(s). Any review or distribution
> > by others is strictly prohibited. If you are not the intended
> > recipient, please contact the sender and delete all copies.
> >
> >
> 
> 
> --
> Patrick Shirkey
> Boost Hardware Ltd
> _______________________________________________
> General mailing list
> General at lists.tizen.org
> https://lists.tizen.org/listinfo/general
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.




More information about the General mailing list