Commit Graph

14 Commits (12844f67c5d746e267756a10b2056db9d719e4ff)

Author SHA1 Message Date
Mooneer Salem 12844f67c5
Second pass at pipeline optimization. (#975)
* Don't write UT features until the end of TX/RX.

* Attempt 2 at preventing spurious Linux asan failures.

* Add PR #975 to changelog.

* Revert "Attempt 2 at preventing spurious Linux asan failures."

This reverts commit 8f53d1a980.

* Another attempt at preventing spurious asan failures.

* Revert back to k6aq.net hosted wheels.

* Resolve warnings in pipeline tests.

* ResampleStep: skip processing if no samples available.

* TapStep: execute in separate thread.

* Fix issue causing mic audio to stop when using Speex.

* Add a bit more time between loads.

* Use -O3 for libsamplerate to see if that helps.

* Revert "Add a bit more time between loads."

This reverts commit 1273d6531e.

* Disable content caching during CI build.

* Add printout of sysctl for debugging.

* Try speexdsp resampling to see if we can reduce CPU usage.

* Slightly increase quality.

* Increase quality again.

* Fix monitor functions not producing audio.

* Reenable Instruments tracing.

* Experiment: update librade to grab sync and SNR while running RX.

* Update SNR retrieval to make sure it happens on RX thread.

* Bump up resample quality again.

* Go back to libsamplerate.

* Use same semaphore wait time calculation for macOS as with other platforms.

* Try 20ms time quantum.

* Try 40ms.

* Back to 60ms.

* Wait needs to be based on start of work.

* Revert "Go back to libsamplerate."

This reverts commit 86d1c640ff.

* Try maximum quality on speex resampler first.

* Go back to calling fwrite() immediately on FIFO read.

* Revert resampler changes, again.

* Try experimental RADE branch again.

* Out of left field: try 2.9ms quantum like what Chrome uses.

* Improve reliability of Linux rade_loss test.

* Smooth out longer than expected semaphore waits on macOS.

* Fix compiler errors.

* Put quantum back to 60ms.

* Fix compiler error.

* More experimentation with time quantums.

* Use relaxed atomic semantics for macOS worker counter.

* Back to 2.9ms due to mpp test failures.

* Disable processing stats.

* Time quantum should match buffer size.

* Remove unneeded variable.

* Time quantum was accidentally getting set to 0.

* Try to reduce standard deviation for macOS wait time.

* Revert previous change, try just halving the wait time.

* Try to reduce likelihood of scheduler contention on macOS.

* macOS: Set lower thread QoS for non-audio threads.

* Using infinite wait for ThreadedObject.

* Force lowest possible latency.

* We can reference the activity ID directly.

* Avoid crash on stop from previous changes.

* Revert recent PulseAudioDevice changes in case those are causing 22.04 tests to fail.

* Try different macOS flag to improve responsiveness.

* Remove memset call from TX handling.

* Disable real-time threading on macOS.

* Use setiopolicy_np() to make sure writing RADE features doesn't interfere with the rest of the app.

* Resample input file for rade_loss test to lower CPU usage.

* Try soxr resampler for further CPU usage reductions.

* Forgot missed file.

* On macOS CI runs, we should also use the Homebrew soxr package.

* Revert soxr adoption, does not meet real time safety guidelines.

* Try deferring feature file writes until the end again.

* Turn on PYTHONOPTIMIZE.

* Read/write entire blocks from FIFO, not just single samples.

* Warning cleanup.

* Allow partial output if FIFO gets empty.

* Remove unneeded memset().

* Fix race condition preventing user from properly being hidden from FreeDV Reporter on startup.

* Downgrade additional pipeline threads to Utility QoS.

* Try malloc-free FIFO for RADE steps.

* Use 2048 samples instead.

* Use PreAllocatedFIFO in RADE steps to avoid cache misses.

* Use MKL enabled version of numpy for Intel macOS.

* Try Intel MKL version of PyTorch for macOS.

* Fix GH compiler issue on macOS ARM.

* Replace sleep_for with semaphore for TapStep.

* Only signal semaphore every 100ms like with previous sleep_based impl.

* Replace additional sleeps with semaphore waits.

* Fix compiler error.

* Adjust timing thresholds for dumping feature debug data.

* macOS: reenable real-time threading as a test.

* Make sure both RX and TX threads start processing at the same time.

* Update missed shared_ptr reference.

* Remove hidden allocation due to use of lambdas.

* Undo FIFO size reduction.

* Suppress spurious PlaybackStep completion messages.

* Fix CI compiler error.

* Use Time Profiler to get a better idea as to why RADE steps intermittently take longer to execute.

* g_voice_keyer_tx should use acquire/release.

* Revert "Use Time Profiler to get a better idea as to why RADE steps intermittently take longer to execute."

This reverts commit 711f83be0c.

* Only grab TX status once through the GUI timer.

* Only get sync once per timer run.

* Remove use of wxThread from TxRxThread.

* Use relaxed for retrieving sync status.

* Test: disable Mic In plot.

* Try using relaxed memory ordering.

* Revert "Use relaxed for retrieving sync status."

This reverts commit 364bc62e8c.

* Revert "Try using relaxed memory ordering."

This reverts commit bac5a8a799.

* One more attempt at using Speex resampler.

* Add optimizations for PlotScalar in case it's impacting test runs.

* Reenable lower quality level for plots.

* Try quality level 6.

* Actually fix spurious playback complete log messages.

* Back to quality level 10.

* macOS: Begin wait from end of work.

* Try DISPATCH_TIME_FOREVER.

* Revert "Try DISPATCH_TIME_FOREVER."

This reverts commit a913e8573e.

* Revert "macOS: Begin wait from end of work."

This reverts commit 7c719a3e3c.

* Add crazy hack to inhibit the UI thread while the pipeline is executing.

* Try checking pipeline spinlock every time we have to redraw something.

* Try actually sleeping instead of yielding.

* Revert crazy hack.

* Use separate RX thread instead of ThreadedTimer.

* Begin splitting GUI timer work into separate timers.

* Only update sync and TX state on the timers that need it.

* Defer item updates until highlights are updated.

* Delete accidentally added files.

* Try smaller block sizes.

* Try bigger block size for comparison.

* Re-disable real-time threading.

* Preallocate what we need to output RX/TX features.

* No need to duplicate bitmap to draw onto graphics context. (Reduces number of mmap() calls req'd on macOS.)

* Reenable real time threading.

* Force 50ms time quantum regardless of buffer size.

* Update reporting timers to attempt to guarantee that only one is active at a time.

* Add safety offset to BlackHole virtual audio device.

* WIP: Reduce CPU usage in PlotWaterfall.

* Fix compiler errors and rendering bugs.

* Fix compile error for real this time.

* Don't update FreeDV Reporter window if it's not actually visible.

* Back to 60ms time quantum.

* Back to 50ms.

* Slightly adjust how IsShownOnScreen() suppresses updates.

* Just use an array instead of map for PlotScalar.

* Prevent propagation of changes to FreeDV Reporter window unless something actually changed.

* Tweak Speex resampler quality again.

* Resample level 7.

* Don't need to recreate ThreadedTimer thread every time we need to restart.

* Try not running xctrace to see if we're close to fixing the issues.

* Wait a bit more time between loop iterations when playing UT recordings.

* Revert "Try not running xctrace to see if we're close to fixing the issues."

This reverts commit decff7a5b2.

* Add Time Profiler to xctrace files.

* Make all FreeDV Reporter rows invisible when window is not visible.

* Make FreeDV Reporter object synchronous.

* Add support for queuing on parent object thread instead of own.

* Fix Linux segfault.

* Use macOS Dispatch for ThreadedObject.

* Try disabling threading in TapStep.

* Reenable threading for TapStep.

* More quality level tweaking.

* Re-disable separate threading.

* Only allocate memory for feature storage if necessary.

* Use GCD for TapStep.

* Fix compiler error on Linux.

* Tweak maximum sizes of FIFOs based on recorded max usage during ctests.

* Additional adjustments to FIFO sizes.

* More tweaks of FIFO sizes for Linux.

* Disable workflow telemtry for macOS runs.

* Fix syntax error.

* Revert "Use GCD for TapStep."

This reverts commit e1522a7ab3.

* Fix memory violations when using PlotScalar.

* Fix rendering bugs from previous optimization.

* Try only returning numInputSamples at a time from vocoders.

* Revert src/pipeline to last passed build.

* Quality level 8 works better locally.

* Try 40ms blocks on macOS.

* Reduce the number of context switches while handling FreeDV Reporter traffic.

* Try reducing FIFO sizes again.

* Try reverting PyTorch upgrade.

* Didn't fully revert PyTorch changes.

* SoX is ignoring errors from Core Audio, use buffer sizes known to work.

* SoX divides by 8 to produce frame size sent to CoreAudio.

* Reset resampler back to last known working level.

* Update Linux/macOS audio code to match Windows implementation more closely.

* Try 40ms blocks again.

* Add middle click reset support for speaker/mic level.

* Revert "Try 40ms blocks again."

This reverts commit 3ca16e0335.

* Have FreeDV itself load test.wav due to performance limitations in GH environment.

* GitHub Actions: Disable iCloud (!)

* Need to set config value and not the slider (middle-click).

* Disable xctrace collection.

* Disable mic/speaker level slider unless modem is active.

* Return to libsamplerate for ResampleStep.

* MacAudioDevice cleanup.

* Remove unneeded warning message.

* Need memory_order_acquire for legacy sync state.

* Try RADE main branch again.

* Debugging: see if anything else is running that shouldn't be.

* GitHub: disable Spotlight too.

* Fix typo in README.

* Reenable macOS runner debugging.

* Include mimalloc for a bit extra boost.

* Disable mimalloc ctests.

* Disable macOS debugging again.

* mimalloc wasn't actually being used on macOS.

* mimalloc: decrease purge interval.

* Fix hidden memory allocation during TX output handling.

* Fix additional hidden memory allocations.

* Unit test: make sure rade_loss shows sync state.

* More hidden memory allocation removals.

* Further reduce duplicate memory allocations.

* Fix voice keyer bug reported by @barjac.

* More cleanup of unneeded allocations.

* Fix mini versions of scalar plots (discovered during testing).

* Remove some wxAsyncCallEventFunctor instances from gprof profile.

* Fix compiler error.

* Additional refactoring and memory allocation reduction.

* Extend purge delay a bit just in case.

* Allow GitHub Actions to re-execute test in case of an intermittent failure.

* Fix typo in rade_reporting test.

* Minor reduction in allocations during FreeDV Reporter RX handling.

* Use std::atomic for PlaybackStep.

* Revert "Use std::atomic for PlaybackStep."

This reverts commit 0ee65a38eb.

* Return nullptr to PlaybackStep if not actively playing.

* PlaybackStep: wait for main thread to finish stopping TX instead of previous approach.

* Fix rendering issue going between TX and RX.

* Fix issue where VK monitor plays previously TX audio prior to current VK cycle.

* Fix segfault in Hamlib on pressing Stop.

* Add missed destroying check while setting PTT.

* Additional cleanup in rig_control due to previous commits.
2025-08-30 09:18:53 -07:00
Mooneer Salem 919bba2c14
Further audio dropout improvements (#876)
* Adjust constants based on Instruments analysis.

* Update comments.

* Try reading only FRAME_DURATION at a time during TX.

* Speex step doesn't need to be locked.

* Use shared pointers instead of locking during EqualizerStep.

* Move file read out of the RT context.

* Remove file I/O from RT context for RecordStep too.

* Try requesting 10ms blocks from Windows.

* Allow all GH tests to run.

* Revert back to 20ms frames.

* Minor tweak to Easy Setup window formatting.

* Revert "Revert back to 20ms frames."

This reverts commit b67313cdef.

* Update freedv-ctest-fullduplex.conf.tmpl

Disable multiple RX for testing.

* Turn off multiple RX for reporting tests as well.

* Force single threaded if only one step is available.

* Revert changes to freedv-ctest-fullduplex.conf.tmpl.

* Revert changes to freedv-ctest-reporting.conf.tmpl.

* Need to wake up all threads whenever we get a block of audio.

* Fix compiler errors.

* Revert previous changes as they didn't help.

* Refactor ParallelStep to remove usage of locks.

* Fix compiler errors and warnings.

* Fix issue preventing EOO from being sent.

* Another attempt at fixing the test failures.

* Use semaphores to wake up ParallelStep threads on receipt of new audio data.

* Fix compiler errors.

* Forgot to wake up helper threads.

* Helper threads should wait until ended or until there's data.

* Fix compiler error.

* Increase fifo sizes for FreeDV TX and RX steps.

* macOS: add protection in case semaphore wasn't created.

* Prevent waterfall from clearing itself when RX mode changes.

* Add logic to read in any data that may have come in during processing.

* GitHub Actions: run all Windows tests even if one or more fail.

* Revert "Add logic to read in any data that may have come in during processing."

This reverts commit 11ce99bb6a.

* Meter out only a little bit of data at a time.

* Revert "Meter out only a little bit of data at a time."

This reverts commit 35e81d6ce5.

* Use 20ms blocks instead of 10ms.

* Revert "Use 20ms blocks instead of 10ms."

This reverts commit 1ec6fb308a.

* Update FRAME_DURATION instead.

* Only process entire TX blocks at a time.

* Set sRGB color space for all open windows.

* Change version tag to rc to reflect release candidate status.

* Revert waveform FIFO changes from previous PR.

* libsamplerate needs to be built with optimizations on macOS and Windows.

* Fix Windows build failure.

* Update LLVM MinGW in Windows CI build to match version used for packaging.

* Test: use RADE main to see if Python GC actually matters.
2025-05-19 11:56:05 -07:00
Mooneer Salem 7cdb9e8a7d
Improve EOO TX/RX reliability (#855)
* Create FIFOs with correct sizes to reduce EOO time.

* Try using configured fifo size for all FIFOs.

* Use correct variable names when definining FIFO sizes.

* Update wxWidgets to 3.2.7 to see if that fixes the build failure.

* Use Ubuntu 24.04 for Windows build to match Linux.

* Patch libsamplerate to avoid GH action failure.

* Need to remove 3.1 from samplerate-cmake.patch

* Fix issue causing cmake to hang if being rerun.

* Temporarily use wxWidgets master as latest Xcode can't build 3.2.7.

* Add support for rig control during RADE EOO ctests.

* Try adding a bit of time to see if it'll decode.

* Fix GH action failure.

* Fix Windows build failure.

* Use same delay code as other existing similar logic.

* Add 20ms delay in mock rigctld to better match behavior with actual hardware.

* Fix Windows build issue for real.

* Fix samplerate patch issue.

* Include wxWidgets manifest code since 3.3 will soon require it.

* Use 3.2 branch as we can build on macOS now in that branch.

* Add IAudioDevice function to allow retrieval of device latency.

* Try shrinking the number of samples FreeDVInterface returns for RADE.

* Add missed get() call in PortAudio logic.

* We really shouldn't need to add txIn latency.

* We need to see the logs from when TX happens during ctest.

* Explicitly disable power savings for audio (macOS).

* Allow partial reads from TX output FIFO.

* First pass at calculating latency for WASAPI.

* Try IO frame size of 1024 to improve pass rate of GH Actions.

* Initial implementation of RADE reporting test on Windows.

* Remove unneeded flag from previously added script.

* Fix various issues with PS script.

* Revert "Try IO frame size of 1024 to improve pass rate of GH Actions."

This reverts commit 1161d9505d.

* Use FDV output, not mock rigctl output, for comparison.

* Use GetStreamLatency() instead.

* Add logging to help determine why WASAPI latency is incorrect.

* Need GetDevicePeriod as well for fully accurate latency measurements.

* Buffer size is the minimum bound on latency. Or at least it seems like it would be, anyway.

* Guarantee that we have universal macOS binary even if tests fail.

* Also take into account PTT response time (i.e. for SDRs).

* Only need to add half of the rig response time for good results.

* Forgot implementation of getRigResponseTimeMicroseconds() for OmniRig.

* Prevent negative zero SNRs from appearing in GUI.

* Try smallest buffer size possible for macOS audio.

* Fix macOS compiler error.

* (Windows) Use event based triggering to provide audio to/from FreeDV.

* Divide by number of channels to get actual latency on macOS.

* Increase minimum frame size to 128 on macOS.

* Oops, types need to be the same.

* Fix deadlock in Windows audio from previous commits.

* Try 256 buffer size on macOS.

* Use minimum of 40ms latency on macOS and Windows.

* No need for the samplerate patch anymore.

* Fix comments.
2025-04-11 19:33:33 -07:00
Mooneer Salem e462cd4060 Ignore libsamplerate UT. 2024-01-14 22:08:26 -08:00
Mooneer Salem 0144bf9e2b Fix build issue due to recent samplerate changes. 2023-01-15 02:16:25 -08:00
Mooneer Salem 944dfcf090 Include samplerate in the package search paths. 2022-12-04 09:33:53 -08:00
Mooneer Salem 9ef899fdd5 Additional cleanup and support for LLVM MinGW. 2022-11-26 08:30:45 -08:00
Mooneer Salem 126c3879da Add missed BUILD_OSX_UNIVERSAL checks. 2020-11-15 21:35:20 -08:00
Mooneer Salem 6261204098 Replace arm64e->arm64. 2020-08-09 17:48:31 -07:00
Mooneer Salem 1d862f8944 Build ARM binary for macOS. 2020-08-02 23:01:07 -07:00
Mooneer Salem a60ffe6c8b Have OSX build staticly link everything but Codec2 and LPCNet. This is needed to guarantee minimum OSX version of 10.9 and usage of libc++ by wxWidgets. 2019-08-11 03:42:14 -07:00
hobbes1069 06a95bec95 Other buildsystem updates for cross-compiling. 2018-05-10 22:47:01 +00:00
drowe67 dc8b158970 builds and runs OK on Linux with internal sox library files, speaker eq tested, README updated for no sox, modfied positioningof audio setup dialog so OK not hidden off screen 2017-01-28 20:51:08 +00:00
hobbes1069 a362d8ebe7 Rename fdmdv2 to freedv. 2015-08-13 13:07:43 +00:00