Enable color emojis in the FreeDV Reporter window (#861)

* WIP patch to wxWidgets to enable color emojis on Windows.

* WIP rewrite of FreeDV Reporter window to use wxDataViewCtrl.

* Apparently we need to associate the model before adding columns.

* Force render the Msg column using Direct2D, causing emojis to display in color.

* Move ReportMessageRenderer to a separate file.

* Only use ReportMessageRenderer on Windows. Also, make sure we don't show ellipsis unless for the Msg field.

* Bring back preservation of sort column.

* Bring back preservation of Msg column width.

* Support Request QSY functionality again.

* Fix rendering issues on macOS.

* Fix Windows compiler errors.

* Fix rendering issue on Windows from previous commits.

* Use correct selection foreground color when rendering with Direct2D.

* Set minimum column widths.

* Column headings should be centered.

* Batch up item changed notifications during refresh.

* Reenable clipboard and tooltip support.

* Ellipsize message text on Windows.

* Allow tooltips to at least show when clicking on a row.

* Have emojis display in color in tooltips as well.

* Add warning message when not building wxWidgets ourselves.

* Only notify of adding new connection if visible.

* Add additional checks around ItemAdded/ItemDeleted.

* Add/remove items individually every refresh to avoid intermittent Linux crashes.

* Defer memory deallocation until after the UI actually removes the item from view.

* Another attempt at fixing the crash.

* Need to validate pointer in GetAttr() too.

* Undo previous changes, try locking allReporterData_ instead.

* Fix Hamlib compiler failure due to changes in 4.6.

* Add some missed locks.

* Replace locks with assertions that we're on the main thread.

* Probably don't need to defer delete.

* Add additional checking around query methods.

* Warning cleanup.

* Make sure items are unselected before deleting.

* Unselect selected user once QSY request has been sent.

* Force a complete redraw when rows go away.

* Try another way of forcing a redraw.

* We only need to call Cleared() once per row.

* Improve CPU usage.

* Aggressively clear selected users when not actively interacting with them.

* Sort by connect time if no columns are selected for sorting.

* PAIn thread should be RT too (missed from previous PR).

* Force a re-sort on connect.

* Resort on every update from the server.

* Sort fix that hopefully doesn't crash.

* Some minor additional changes.

* Disable PAIn thread and just rely on the normal PulseAudio callbacks for passing audio in and out.

* Make re-sorting happen only once a second.

* Try putting mutexes around all data access.

* Increase RLIMIT_RTTIME just in case.

* Add missed locks.

* RT limit was set too high.

* Sleep for less time than latency if possible.

* Fix crash in TapStep while stopping.

* Defer FreeDV Reporter initialization on startup to make sure GUI is up beforehand.

* Try again to get RTKit to work in the GH environment.

* Try capping maxlength to avoid RX FIFO full errors.

* Increase max buffer size due to overrun errors.

* Default maxlength for output devices to prevent overruns.

* Try bringing back PAin thread. WIP

* Add missed PulseAudio locking.

* Remove more missed mallocs from RT path.

* Remove unneeded code from PulseAudio logic.

* Tab cleanup.

* Increase RX FIFO size.

* Force sleep if we take much longer than expected (avoids kill by rtkit).

* Forgot to check in file.

* Fix compiler error for real.

* Fix issue causing occasionally being unable to get the mic input on macOS.

* Add timeout for EOO in case we lose input devices.

* Add additional protection to prevent the OS from killing the app.

* macOS: get device names an alternate way to avoid empty names for certain devices.

* Make RX plots more reliable when pipewire sends multiple seconds of RX audio at once.

* Oops, used wrong comparison for EOO timeout.

* Catch SIGXCPU in case we exceed RT limits.

* Prevent RX processing from occurring if there's no space in the output FIFO.

* Apparently I was setting the soft and hard rlimits to be the same.

* Lower soft RT limit to 10ms.

* Disable RT scheduling on Linux.

* Added additional protection against server-caused race condition.

* Defer add to table until after we get connection successful message.

* Add missed lock on connect successful message.

* Add missed visibility check.

* Fix issue on Linux.

* Add additional logging to help diagnose duplicate callsign issue.

* Downgrade additional logging to debug/verbose only.

* Fix bug where incorrect SNR was being reported for RADE signals after EOO received.

* Back out previous change buffering rows until fully connected as it doesn't work properly on Windows.

* Fix right-click menu regression.

* Fix issue preventing column sorting on Windows.

* Upgrade wxWidgets to 3.2.8.

* Right-click should not select items.

.github/workflows/cmake-linux.yml

@@ -20,13 +20,19 @@ jobs:
     steps:
     - uses: actions/checkout@v4
 
-    - name: Install packages
+    - name: Install rtkit for RT threading
       shell: bash
       run: |
            sudo apt-get update
            sudo apt-get upgrade -y
-           sudo apt-get install codespell libpulse-dev libspeexdsp-dev libsamplerate0-dev sox git libwxgtk3.2-dev portaudio19-dev libhamlib-dev libasound2-dev libao-dev libgsm1-dev libsndfile-dev xvfb pipewire pulseaudio-utils pipewire-pulse wireplumber metacity dbus-x11 at-spi2-core rtkit octave octave-signal libdbus-1-dev
+           sudo apt-get install dbus-x11 rtkit libdbus-1-dev polkitd
-           sudo usermod -a -G rtkit $(whoami)
+           sudo sed -i 's/no/yes/g' /usr/share/polkit-1/actions/org.freedesktop.RealtimeKit1.policy 
+           sudo systemctl restart polkit
+
+    - name: Install packages
+      shell: bash
+      run: |
+           sudo apt-get install codespell libpulse-dev libspeexdsp-dev libsamplerate0-dev sox git libwxgtk3.2-dev portaudio19-dev libhamlib-dev libasound2-dev libao-dev libgsm1-dev libsndfile-dev xvfb pipewire pulseaudio-utils pipewire-pulse wireplumber metacity at-spi2-core octave octave-signal
             
     - name: Spellcheck codebase
       shell: bash

CMakeLists.txt

@@ -243,6 +243,14 @@ if(NOT BOOTSTRAP_WXWIDGETS)
         if(WX_VERSION VERSION_EQUAL ${WX_VERSION_MIN}
            OR WX_VERSION VERSION_GREATER ${WX_VERSION_MIN})
             message(STATUS "wxWidgets version: ${WX_VERSION}")
+            if(WIN32 OR APPLE)
+                message(WARNING 
+                    "On Windows and macOS, FreeDV patches wxWidgets to ensure consistent UI behavior across platforms."
+                    "It is highly recommended to allow FreeDV to build its own wxWidgets with these patches by setting "
+                    "-DBOOTSTRAP_WXWIDGETS=TRUE instead of using any version of wxWidgets already on the system. "
+                    "Alternatively, you can apply the patch in cmake/wxWidgets-Direct2D-color-font.patch to the wxWidgets "
+                    "source tree and build it yourself.")
+            endif(WIN32 OR APPLE)
         else()
             set(BOOTSTRAP_WXWIDGETS TRUE)
         endif()

USER_MANUAL.md

@@ -916,7 +916,7 @@ LDPC | Low Density Parity Check Codes - a family of powerful FEC codes
     * Shorten PulseAudio/pipewire app name. (PR #843)
 3. Build system:
     * Allow overriding the version tag when building. (PR #727)
-    * Update wxWidgets to 3.2.6. (PR #748)
+    * Update wxWidgets to 3.2.8. (PR #861)
     * Update Hamlib to 4.6.2. (PR #834)
     * Use optimal number of parallel builds during build process. (PR #842)
 4. Miscellaneous:

cmake/BuildWxWidgets.cmake

@@ -1,4 +1,4 @@
-set(WXWIDGETS_VERSION "3.2.7")
+set(WXWIDGETS_VERSION "3.2.8")
 
 # Ensure that the wxWidgets library is staticly built.
 set(wxBUILD_SHARED OFF CACHE BOOL "Build wx libraries as shared libs")
@@ -19,14 +19,19 @@ set(wxUSE_LIBSDL OFF CACHE STRING "use SDL for audio on Unix")
 set(wxUSE_LIBMSPACK OFF CACHE STRING "use libmspack (CHM help files loading)")
 set(wxUSE_LIBICONV OFF CACHE STRING "disable use of libiconv")
 
+if(WIN32)
+set(wxUSE_GRAPHICS_DIRECT2D ON CACHE STRING "use Direct2D graphics context")
+endif(WIN32)
+
 include(FetchContent)
 FetchContent_Declare(
     wxWidgets
     GIT_REPOSITORY https://github.com/wxWidgets/wxWidgets.git
     GIT_SHALLOW    TRUE
     GIT_PROGRESS   TRUE
-    #GIT_TAG        v${WXWIDGETS_VERSION}
+    GIT_TAG        v${WXWIDGETS_VERSION}
-    GIT_TAG 3.2
+    PATCH_COMMAND  git apply ${CMAKE_SOURCE_DIR}/cmake/wxWidgets-Direct2D-color-font.patch
+    UPDATE_DISCONNECTED 1
 )
 
 FetchContent_GetProperties(wxWidgets)

cmake/wxWidgets-Direct2D-color-font.patch

@@ -0,0 +1,129 @@
+diff --git a/include/wx/msw/setup.h b/include/wx/msw/setup.h
+index b1d20d927..4f834c941 100644
+--- a/include/wx/msw/setup.h
++++ b/include/wx/msw/setup.h
+@@ -1648,7 +1648,7 @@
+ #if defined(_MSC_VER) && _MSC_VER >= 1600
+     #define wxUSE_GRAPHICS_DIRECT2D wxUSE_GRAPHICS_CONTEXT
+ #else
+-    #define wxUSE_GRAPHICS_DIRECT2D 0
++    #define wxUSE_GRAPHICS_DIRECT2D wxUSE_GRAPHICS_CONTEXT
+ #endif
+ 
+ // wxWebRequest backend based on WinHTTP.
+diff --git a/src/generic/tipwin.cpp b/src/generic/tipwin.cpp
+index f8ac37f5f..c95371d06 100644
+--- a/src/generic/tipwin.cpp
++++ b/src/generic/tipwin.cpp
+@@ -33,6 +33,10 @@
+ #include "wx/display.h"
+ #include "wx/vector.h"
+ 
++#if defined(WIN32)
++#include "wx/graphics.h"
++#endif // defined(WIN32)
++
+ // ----------------------------------------------------------------------------
+ // constants
+ // ----------------------------------------------------------------------------
+@@ -298,15 +302,38 @@ void wxTipWindowView::OnPaint(wxPaintEvent& WXUNUSED(event))
+     rect.width = size.x;
+     rect.height = size.y;
+ 
+-    // first filll the background
+-    dc.SetBrush(wxBrush(GetBackgroundColour(), wxBRUSHSTYLE_SOLID));
+-    dc.SetPen(wxPen(GetForegroundColour(), 1, wxPENSTYLE_SOLID));
+-    dc.DrawRectangle(rect);
++#if defined(WIN32)
++    // Tooltips should be rendered with Direct2D if at all possible.
++    wxGraphicsRenderer* renderer = wxGraphicsRenderer::GetDirect2DRenderer();
++    wxGraphicsContext* context = nullptr;
++    if (renderer != nullptr)
++    {
++        context = renderer->CreateContextFromUnknownDC(dc);
++        if (context != nullptr)
++        {
++            // first fill the background
++            context->SetBrush(wxBrush(GetBackgroundColour(), wxBRUSHSTYLE_SOLID));
++            context->SetPen(wxPen(GetForegroundColour(), 1, wxPENSTYLE_SOLID));
++            context->DrawRectangle(0, 0, rect.width - 1, rect.height - 1);
+ 
+-    // and then draw the text line by line
+-    dc.SetTextBackground(GetBackgroundColour());
+-    dc.SetTextForeground(GetForegroundColour());
+-    dc.SetFont(GetFont());
++            // and then draw the text line by line
++            context->SetFont(GetFont(), GetForegroundColour());
++        }
++    }
++
++    if (context == nullptr)
++#endif // defined(WIN32)
++    {
++        // first filll the background
++        dc.SetBrush(wxBrush(GetBackgroundColour(), wxBRUSHSTYLE_SOLID));
++        dc.SetPen(wxPen(GetForegroundColour(), 1, wxPENSTYLE_SOLID));
++        dc.DrawRectangle(rect);
++
++        // and then draw the text line by line
++        dc.SetTextBackground(GetBackgroundColour());
++        dc.SetTextForeground(GetForegroundColour());
++        dc.SetFont(GetFont());
++    }
+ 
+     wxPoint pt;
+     pt.x = TEXT_MARGIN_X;
+@@ -314,10 +341,26 @@ void wxTipWindowView::OnPaint(wxPaintEvent& WXUNUSED(event))
+     const size_t count = m_textLines.size();
+     for ( size_t n = 0; n < count; n++ )
+     {
+-        dc.DrawText(m_textLines[n], pt);
++#if defined(WIN32)
++        if (context != nullptr)
++        {
++            context->DrawText(m_textLines[n], pt.x, pt.y);
++        }
++        else
++#endif // defined(WIN32)
++        {
++            dc.DrawText(m_textLines[n], pt);
++        }
+ 
+         pt.y += m_heightLine;
+     }
++
++#if defined(WIN32)
++    if (context != nullptr)
++    {
++        delete context;
++    }
++#endif // defined(WIN32)
+ }
+ 
+ void wxTipWindowView::OnMouseClick(wxMouseEvent& WXUNUSED(event))
+diff --git a/src/msw/graphicsd2d.cpp b/src/msw/graphicsd2d.cpp
+index 89b74102a..e2a96a760 100644
+--- a/src/msw/graphicsd2d.cpp
++++ b/src/msw/graphicsd2d.cpp
+@@ -4767,7 +4767,8 @@ void wxD2DContext::DoDrawText(const wxString& str, wxDouble x, wxDouble y)
+     GetRenderTarget()->DrawTextLayout(
+         D2D1::Point2F(x, y),
+         textLayout,
+-        fontData->GetBrushData().GetBrush());
++        fontData->GetBrushData().GetBrush(),
++        D2D1_DRAW_TEXT_OPTIONS::D2D1_DRAW_TEXT_OPTIONS_ENABLE_COLOR_FONT);
+ }
+ 
+ void wxD2DContext::EnsureInitialized()
+diff --git a/src/osx/cocoa/dataview.mm b/src/osx/cocoa/dataview.mm
+index 93554f1c7..3e879b3c1 100644
+--- a/src/osx/cocoa/dataview.mm
++++ b/src/osx/cocoa/dataview.mm
+@@ -1573,6 +1573,7 @@ outlineView:(NSOutlineView*)outlineView
+         [self setDraggingSourceOperationMask:NSDragOperationEvery forLocal:NO];
+         [self setDraggingSourceOperationMask:NSDragOperationEvery forLocal:YES];
+         [self setTarget:self];
++        self.intercellSpacing = NSZeroSize;
+     }
+     return self;
+ }

src/audio/IAudioDevice.h

@@ -68,6 +68,10 @@ public:
     // Reverts real-time priority for current thread.
     virtual void clearHelperRealTime() override { /* empty */ }
 
+    // Returns true if real-time thread MUST sleep ASAP. Failure to do so
+    // may result in SIGKILL being sent to the process by the kernel.
+    virtual bool mustStopWork() override { return false; }
+
     // Sets user friendly description of device. Not used by all engines.
     void setDescription(std::string desc);
     

src/audio/MacAudioDevice.mm

@@ -136,6 +136,8 @@ void MacAudioDevice::start()
             kAudioDevicePropertyScopeOutput;
         
         Float64 sampleRateAsFloat = sampleRate_;
+
+        log_info("Attempting to set sample rate to %f for device %d", sampleRateAsFloat, coreAudioId_);
         OSStatus error = AudioObjectSetPropertyData(
             coreAudioId_,
             &propertyAddress,
@@ -255,9 +257,10 @@ void MacAudioDevice::start()
                 return OSStatus(noErr);
             };
             
+            AVAudioFormat* inputFormat = [inNode inputFormatForBus:0];
             AVAudioSinkNode* sinkNode = [[AVAudioSinkNode alloc] initWithReceiverBlock:block];
             [engine attachNode:sinkNode];    
-            [engine connect:[engine inputNode] to:sinkNode format:nil]; 
+            [engine connect:inNode to:sinkNode format:inputFormat]; 
         }
         else
         {
@@ -301,10 +304,12 @@ void MacAudioDevice::start()
         [engine prepare];
         if (![engine startAndReturnError:&nse])
         {
+            NSString* errorDesc = [nse localizedDescription];
+            std::string err = std::string("Could not start AVAudioEngine: ") + [errorDesc cStringUsingEncoding:NSUTF8StringEncoding];
+            log_error(err.c_str());
+
             if (onAudioErrorFunction)
             {
-                NSString* errorDesc = [nse localizedDescription];
-                std::string err = std::string("Could not start AVAudioEngine: ") + [errorDesc cStringUsingEncoding:NSUTF8StringEncoding];
                 onAudioErrorFunction(*this, err, onAudioErrorState);
             }
             [engine release];

src/audio/MacAudioEngine.cpp

@@ -26,6 +26,8 @@
 
 #include <CoreAudio/CoreAudio.h>
 
+static const int kAdmMaxDeviceNameSize = 128;
+
 void MacAudioEngine::start()
 {
     // empty - no initialization needed.
@@ -274,9 +276,11 @@ AudioDeviceSpecification MacAudioEngine::getAudioSpecification_(int coreAudioId,
     OSStatus status = noErr;
     
     // Get device name
-    propertyAddress.mSelector = kAudioDevicePropertyDeviceNameCFString;
+    propertyAddress.mSelector = kAudioDevicePropertyDeviceName;
-    propertySize = sizeof(CFStringRef);
+
-    CFStringRef name = nullptr;
+    char name[kAdmMaxDeviceNameSize];
+    propertySize = sizeof(name);
+
     status = AudioObjectGetPropertyData(
         coreAudioId,
         &propertyAddress,
@@ -294,8 +298,7 @@ AudioDeviceSpecification MacAudioEngine::getAudioSpecification_(int coreAudioId,
         return AudioDeviceSpecification::GetInvalidDevice();
     }
     
-    std::string deviceName = cfStringToStdString_(name);
+    std::string deviceName = name;
-    CFRelease(name);
     
     // Get HW sample rate
     double sampleRate = 0;
@@ -393,33 +396,3 @@ int MacAudioEngine::getNumChannels_(int coreAudioId, AudioDirection direction)
     
     return numChannels;
 }
-
-/**
- * Converts a CFString to a UTF-8 std::string if possible.
- *
- * @param input A reference to the CFString to convert.
- * @return Returns a std::string containing the contents of CFString converted to UTF-8. Returns
- *  an empty string if the input reference is null or conversion is not possible.
- */
-std::string MacAudioEngine::cfStringToStdString_(CFStringRef input)
-{
-    if (!input)
-        return {};
-
-    // Attempt to access the underlying buffer directly. This only works if no conversion or
-    //  internal allocation is required.
-    auto originalBuffer{ CFStringGetCStringPtr(input, kCFStringEncodingUTF8) };
-    if (originalBuffer)
-        return originalBuffer;
-
-    // Copy the data out to a local buffer.
-    auto lengthInUtf16{ CFStringGetLength(input) };
-    auto maxLengthInUtf8{ CFStringGetMaximumSizeForEncoding(lengthInUtf16,
-        kCFStringEncodingUTF8) + 1 }; // <-- leave room for null terminator
-    std::vector<char> localBuffer(maxLengthInUtf8);
-
-    if (CFStringGetCString(input, localBuffer.data(), maxLengthInUtf8, maxLengthInUtf8))
-        return localBuffer.data();
-
-    return {};
-}

src/audio/MacAudioEngine.h

@@ -46,8 +46,6 @@ public:
     virtual std::vector<int> getSupportedSampleRates(wxString deviceName, AudioDirection direction) override;
     
 private:    
-    std::string cfStringToStdString_(CFStringRef input);
-    
     AudioDeviceSpecification getAudioSpecification_(int coreAudioId, AudioDirection direction);
     int getNumChannels_(int coreAudioId, AudioDirection direction);
 };

src/audio/PulseAudioDevice.cpp

@@ -25,6 +25,7 @@
 #include <chrono>
 #include <sched.h>
 #include <sys/resource.h>
+#include <signal.h>
 
 #include "PulseAudioDevice.h"
 
@@ -36,23 +37,17 @@
 
 using namespace std::chrono_literals;
 
-// Optimal settings based on ones used for PortAudio.
+// Target latency. This controls e.g. how long it takes for
-#define PULSE_FPB 256
+// TX audio to reach the radio.
-#define PULSE_TARGET_LATENCY_US 20000
+#define PULSE_TARGET_LATENCY_US 10000
+
+thread_local std::chrono::high_resolution_clock::time_point PulseAudioDevice::StartTime_;
+thread_local bool PulseAudioDevice::MustStopWork_ = false;
 
 PulseAudioDevice::PulseAudioDevice(pa_threaded_mainloop *mainloop, pa_context* context, wxString devName, IAudioEngine::AudioDirection direction, int sampleRate, int numChannels)
     : context_(context)
     , mainloop_(mainloop)
     , stream_(nullptr)
-    , outputPending_(nullptr)
-    , outputPendingLength_(0)
-    , outputPendingThreadActive_(false)
-    , outputPendingThread_(nullptr)
-    , targetOutputPendingLength_(PULSE_FPB * numChannels * 2)
-    , inputPending_(nullptr)
-    , inputPendingLength_(0)
-    , inputPendingThreadActive_(false)
-    , inputPendingThread_(nullptr)
     , devName_(devName)
     , direction_(direction)
     , sampleRate_(sampleRate)
@@ -104,8 +99,8 @@ void PulseAudioDevice::start()
 
     // recommended settings, i.e. server uses sensible values
     pa_buffer_attr buffer_attr; 
-    buffer_attr.maxlength = (uint32_t)-1;
     buffer_attr.tlength = pa_usec_to_bytes(PULSE_TARGET_LATENCY_US, &sample_specification);
+    buffer_attr.maxlength = (uint32_t)-1;
     buffer_attr.prebuf = 0; // Ensure that we can recover during an underrun
     buffer_attr.minreq = (uint32_t) -1;
     buffer_attr.fragsize = buffer_attr.tlength;
@@ -145,136 +140,11 @@ void PulseAudioDevice::start()
     }
     else
     {
-	// Set up semaphore for signaling workers
+        // Set up semaphore for signaling workers
         if (sem_init(&sem_, 0, 0) < 0)
-	{
+        {
             log_warn("Could not set up semaphore (errno = %d)", errno);
         }
-
-        // Start data collection thread. This thread
-        // is necessary in order to ensure that we can 
-        // provide data to PulseAudio at a rate expected
-        // for the actual latency of the sound device.
-        outputPending_ = nullptr;
-        outputPendingLength_ = 0;
-        targetOutputPendingLength_ = PULSE_FPB * getNumChannels() * 2;
-        outputPendingThreadActive_ = true;
-        inputPending_ = nullptr;
-        inputPendingLength_ = 0;
-        if (direction_ == IAudioEngine::AUDIO_ENGINE_IN)
-        {
-            inputPendingThreadActive_ = true;
-            inputPendingThread_ = new std::thread([&]() {
-#if defined(__linux__)
-                pthread_setname_np(pthread_self(), "FreeDV PAIn");
-#endif // defined(__linux__)
-         
-                while(inputPendingThreadActive_)
-                {
-                    auto currentTime = std::chrono::steady_clock::now();
-                    int currentLength = 0;
-
-                    {
-                        std::unique_lock<std::mutex> lk(inputPendingMutex_);
-                        currentLength = inputPendingLength_;
-                    }
-
-                    currentLength = std::min(currentLength, PULSE_FPB * getNumChannels());
-                    if (currentLength > 0)
-                    {
-                        short data[currentLength];
-                        {
-                            std::unique_lock<std::mutex> lk(inputPendingMutex_);
-                            memcpy(data, inputPending_, currentLength * sizeof(short));
-
-                            short* newInputPending = nullptr;
-                            if (inputPendingLength_ > currentLength)
-                            {
-                                newInputPending = new short[inputPendingLength_ - currentLength];
-                                assert(newInputPending != nullptr);
-                                memcpy(newInputPending, inputPending_ + currentLength, (inputPendingLength_ - currentLength) * sizeof(short));
-                            }
-                            delete[] inputPending_;
-                            inputPending_ = newInputPending;
-                            inputPendingLength_ = inputPendingLength_ - currentLength;
-                        }
-
-                        if (onAudioDataFunction)
-                        {
-                            onAudioDataFunction(*this, data, currentLength / getNumChannels(), onAudioDataState);
-                        }
-			sem_post(&sem_);
-
-                        // Sleep up to the number of milliseconds corresponding to the data received
-                        int numMilliseconds = 1000.0 * ((double)currentLength / getNumChannels()) / (double)getSampleRate();
-                        std::this_thread::sleep_until(currentTime + std::chrono::milliseconds(numMilliseconds));
-                    }
-                    else
-                    {
-                        // Sleep up to 20ms by default if there's no data available.
-                        std::this_thread::sleep_until(currentTime + 20ms);
-                    }
-                }
-            });
-            assert(inputPendingThread_ != nullptr);
-        }
-#if 0
-        else if (direction_ == IAudioEngine::AUDIO_ENGINE_OUT)
-        {
-            outputPendingThread_ = new std::thread([&]() {
-#if defined(__linux__)
-                pthread_setname_np(pthread_self(), "FreeDV PAOut");
-#endif // defined(__linux__)
-
-                while(outputPendingThreadActive_)
-                {
-                    auto currentTime = std::chrono::steady_clock::now();
-                    int currentLength = 0;
-
-                    {
-                        std::unique_lock<std::mutex> lk(outputPendingMutex_);
-                        currentLength = outputPendingLength_;
-                    }
-
-                    if (currentLength < targetOutputPendingLength_)
-                    {                        
-                        short data[PULSE_FPB * getNumChannels()];
-                        memset(data, 0, sizeof(data));
-                        
-                        if (onAudioDataFunction)
-                        {
-                            onAudioDataFunction(*this, data, PULSE_FPB, onAudioDataState);
-                        }
-    
-                        {
-                            std::unique_lock<std::mutex> lk(outputPendingMutex_);
-                            short* temp = new short[outputPendingLength_ + PULSE_FPB * getNumChannels()];
-                            assert(temp != nullptr);
-    
-                            if (outputPendingLength_ > 0)
-                            {
-                                memcpy(temp, outputPending_, outputPendingLength_ * sizeof(short));
-    
-                                delete[] outputPending_;
-                                outputPending_ = nullptr;
-                            }
-                            memcpy(temp + outputPendingLength_, data, sizeof(data));
-    
-                            outputPending_ = temp;
-                            outputPendingLength_ += PULSE_FPB * getNumChannels();
-                        }
-                    }
-
-                    // Sleep the required amount of time to ensure we call onAudioDataFunction
-                    // every PULSE_FPB samples.
-                    int sleepTimeMilliseconds = ((double)PULSE_FPB)/((double)sampleRate_) * 1000.0;
-                    std::this_thread::sleep_until(currentTime + 
-                        std::chrono::milliseconds(sleepTimeMilliseconds));
-                }
-            });
-            assert(outputPendingThread_ != nullptr);
-        }
-#endif
     }
 
     pa_threaded_mainloop_unlock(mainloop_);
@@ -295,54 +165,38 @@ void PulseAudioDevice::stop()
         pa_stream_unref(stream_);
 
         stream_ = nullptr;
-
-        outputPendingThreadActive_ = false;
-        if (outputPendingThread_ != nullptr)
-        {
-            outputPendingThread_->join();
-
-            delete[] outputPending_;
-            outputPending_ = nullptr;
-            outputPendingLength_ = 0;
-
-            delete outputPendingThread_;
-            outputPendingThread_ = nullptr;
-        }
-        inputPendingThreadActive_ = false;
-        if (inputPendingThread_ != nullptr)
-        {
-            inputPendingThread_->join();
-
-            delete[] inputPending_;
-            inputPending_ = nullptr;
-            inputPendingLength_ = 0;
-
-            delete inputPendingThread_;
-            inputPendingThread_ = nullptr;
-        }
-
         sem_destroy(&sem_);
     }
 }
 
 int PulseAudioDevice::getLatencyInMicroseconds()
 {
+    pa_threaded_mainloop_lock(mainloop_);
     pa_usec_t latency = 0;
     if (stream_ != nullptr)
     {
         int neg = 0;
         pa_stream_get_latency(stream_, &latency, &neg); // ignore error and assume 0
     }
+    pa_threaded_mainloop_unlock(mainloop_);
     return (int)latency;
 }
 
 void PulseAudioDevice::setHelperRealTime()
 {
+    // XXX: We can't currently enable RT scheduling on Linux
+    // due to unreliable behavior surrounding how long it takes to
+    // go through a single RX or TX cycle. This unreliability is 
+    // likely due to the use of Python for some parts of RADE. Since
+    // timing is so unreliable and due to the fact that Linux actually
+    // kills processes that it deems as using "too much" CPU while in
+    // real-time, it's better just to use normal scheduling for now.
+#if 0
     // Set RLIMIT_RTTIME, required for rtkit
     struct rlimit rlim;
     memset(&rlim, 0, sizeof(rlim));
-    rlim.rlim_cur = 100000ULL; // 100ms
+    rlim.rlim_cur = 10000ULL; // 10ms
-    rlim.rlim_max = rlim.rlim_cur;
+    rlim.rlim_max = 200000ULL; // 200ms
 
     if ((setrlimit(RLIMIT_RTTIME, &rlim) < 0))
     {
@@ -358,49 +212,89 @@ void PulseAudioDevice::setHelperRealTime()
     {
 #if defined(USE_RTKIT)
         DBusError error;
-	DBusConnection* bus = nullptr;
+        DBusConnection* bus = nullptr;
         int result = 0;
 
         dbus_error_init(&error);
-	if (!(bus = dbus_bus_get(DBUS_BUS_SYSTEM, &error)))
+        if (!(bus = dbus_bus_get(DBUS_BUS_SYSTEM, &error)))
-	{
+        {
             log_warn("Could not connect to system bus: %s", error.message);
-	}
+        }
-	else if ((result = rtkit_make_realtime(bus, 0, p.sched_priority)) < 0)
+        else if ((result = rtkit_make_realtime(bus, 0, p.sched_priority)) < 0)
-	{
+        {
-	    log_warn("rtkit could not make real-time: %s", strerror(-result));
+            log_warn("rtkit could not make real-time: %s", strerror(-result));
         }
 
-	if (bus != nullptr)
+        if (bus != nullptr)
-	{
+        {
             dbus_connection_unref(bus);
-	}
+        }
 #else
         log_warn("No permission to make real-time");
 #endif // defined(USE_RTKIT)
     }
+
+    // Set up signal handling for SIGXCPU
+    struct sigaction action;
+    action.sa_flags = SA_SIGINFO;     
+    action.sa_sigaction = HandleXCPU_;
+    sigaction(SIGXCPU, &action, NULL);
+
+    sigset_t signal_set;
+    sigemptyset(&signal_set);
+    sigaddset(&signal_set, SIGXCPU);
+    sigprocmask(SIG_UNBLOCK, &signal_set, NULL);
+#endif // 0
 }
 
+void PulseAudioDevice::startRealTimeWork()
+{
+    StartTime_ = std::chrono::high_resolution_clock::now();
+
+    sleepFallback_ = false;
+    if (clock_gettime(CLOCK_REALTIME, &ts_) == -1)
+    {
+        sleepFallback_ = true;
+    }
+}
 void PulseAudioDevice::stopRealTimeWork()
 {
-    struct timespec ts;
+    if (sleepFallback_)
-
-    if (clock_gettime(CLOCK_REALTIME, &ts) == -1)
     {
         // Fallback to simple sleep.
         IAudioDevice::stopRealTimeWork();
         return;
     }
 
-    ts.tv_nsec += 10000000;
+    auto latency = getLatencyInMicroseconds();
-    ts.tv_sec += (ts.tv_nsec / 1000000000);
+    if (latency == 0)
-    ts.tv_nsec = ts.tv_nsec % 1000000000;
+    {
+        latency = PULSE_TARGET_LATENCY_US;
+    }
 
-    if (sem_timedwait(&sem_, &ts) < 0 && errno != ETIMEDOUT)
+    ts_.tv_nsec += latency * 1000;
+    if (ts_.tv_nsec >= 1000000000)
+    {
+        ts_.tv_sec++;
+        ts_.tv_nsec -= 1000000000;
+    }
+
+    if (sem_timedwait(&sem_, &ts_) < 0 && errno != ETIMEDOUT)
     {
         // Fallback to simple sleep.
         IAudioDevice::stopRealTimeWork();
     }
+    else if (errno == ETIMEDOUT)
+    {
+        auto endTime = std::chrono::high_resolution_clock::now();
+        if ((endTime - StartTime_) >= std::chrono::microseconds(PULSE_TARGET_LATENCY_US * 10))
+        {
+            // Took a lot longer than expected. Force a sleep so we don't get killed by rtkit.
+            std::this_thread::sleep_for(std::chrono::microseconds(PULSE_TARGET_LATENCY_US));
+        }
+    }
+
+    MustStopWork_ = false;
 }
 
 void PulseAudioDevice::clearHelperRealTime()
@@ -408,6 +302,11 @@ void PulseAudioDevice::clearHelperRealTime()
     IAudioDevice::clearHelperRealTime();
 }
 
+bool PulseAudioDevice::mustStopWork()
+{
+    return MustStopWork_;
+}
+
 void PulseAudioDevice::StreamReadCallback_(pa_stream *s, size_t length, void *userdata)
 {
     const void* data = nullptr;
@@ -418,26 +317,14 @@ void PulseAudioDevice::StreamReadCallback_(pa_stream *s, size_t length, void *us
         pa_stream_peek(s, &data, &length);
         if (!data || length == 0) 
         {
-            break;
+            return; //break;
         }
 
-        // Append received audio to pending block
+        if (thisObj->onAudioDataFunction)
         {
-            std::unique_lock<std::mutex> lk(thisObj->inputPendingMutex_);
+            thisObj->onAudioDataFunction(*thisObj, const_cast<void*>(data), length / thisObj->getNumChannels() / sizeof(short), thisObj->onAudioDataState);
-            short* temp = new short[thisObj->inputPendingLength_ + length / sizeof(short)];
-            assert(temp != nullptr);
-    
-            if (thisObj->inputPendingLength_ > 0)
-            {
-                memcpy(temp, thisObj->inputPending_, thisObj->inputPendingLength_ * sizeof(short));
-                delete[] thisObj->inputPending_;
-                thisObj->inputPending_ = nullptr;
-            }
-            memcpy(temp + thisObj->inputPendingLength_, data, length);
-            thisObj->inputPending_ = temp;
-            thisObj->inputPendingLength_ += length / sizeof(short);
         }
-
+        sem_post(&thisObj->sem_);
         pa_stream_drop(s);
     } while (pa_stream_readable_size(s) > 0);
 }
@@ -452,26 +339,6 @@ void PulseAudioDevice::StreamWriteCallback_(pa_stream *s, size_t length, void *u
         memset(data, 0, sizeof(data));
 
         PulseAudioDevice* thisObj = static_cast<PulseAudioDevice*>(userdata);
-#if 0
-        {
-            std::unique_lock<std::mutex> lk(thisObj->outputPendingMutex_);
-            if (thisObj->outputPendingLength_ >= numSamples)
-            {
-                memcpy(data, thisObj->outputPending_, sizeof(data));
-
-                short* tmp = new short[thisObj->outputPendingLength_ - numSamples];
-                assert(tmp != nullptr);
-
-                thisObj->outputPendingLength_ -= numSamples;
-                memcpy(tmp, thisObj->outputPending_ + numSamples, sizeof(short) * thisObj->outputPendingLength_);
-
-                delete[] thisObj->outputPending_;
-                thisObj->outputPending_ = tmp;
-            }
-
-            thisObj->targetOutputPendingLength_ = std::max(thisObj->targetOutputPendingLength_, 2 * numSamples);
-        }
-#endif
 
         if (thisObj->onAudioDataFunction)
         {
@@ -541,3 +408,10 @@ void PulseAudioDevice::StreamLatencyCallback_(pa_stream *p, void *userdata)
     fprintf(stderr, "Current target buffer size for %s: %d\n", (const char*)thisObj->devName_.ToUTF8(), thisObj->targetOutputPendingLength_);
 }
 #endif // 0
+
+void PulseAudioDevice::HandleXCPU_(int signum, siginfo_t *info, void *extra)
+{
+    // Notify thread that it has to stop work immediately and sleep.
+    log_warn("Taking too much CPU handling real-time tasks, pausing for a bit");
+    MustStopWork_ = true;
+}

src/audio/PulseAudioDevice.h

@@ -52,6 +52,9 @@ public:
     // called from the thread that will be operating on received audio.
     virtual void setHelperRealTime() override;
 
+    // Lets audio system know that we're starting work on received audio.
+    virtual void startRealTimeWork() override;
+
     // Lets audio system know that we're done with the work on the received
     // audio.
     virtual void stopRealTimeWork() override;
@@ -59,6 +62,10 @@ public:
     // Reverts real-time priority for current thread.
     virtual void clearHelperRealTime() override;
 
+    // Returns true if real-time thread MUST sleep ASAP. Failure to do so
+    // may result in SIGKILL being sent to the process by the kernel.
+    virtual bool mustStopWork() override;
+
 protected:
     // PulseAudioDevice cannot be created directly, only via PulseAudioEngine.
     friend class PulseAudioEngine;
@@ -70,19 +77,6 @@ private:
     pa_threaded_mainloop* mainloop_;
     pa_stream* stream_;
 
-    short* outputPending_;
-    int outputPendingLength_;
-    bool outputPendingThreadActive_;
-    std::mutex outputPendingMutex_;
-    std::thread* outputPendingThread_;
-    int targetOutputPendingLength_;
-
-    short* inputPending_;
-    int inputPendingLength_;
-    bool inputPendingThreadActive_;
-    std::mutex inputPendingMutex_;
-    std::thread* inputPendingThread_;
-
     wxString devName_;
     IAudioEngine::AudioDirection direction_;
     int sampleRate_;
@@ -90,7 +84,12 @@ private:
     std::mutex streamStateMutex_;
     std::condition_variable streamStateCondVar_;
     
+    thread_local static std::chrono::high_resolution_clock::time_point StartTime_;
+    thread_local static bool MustStopWork_;
+
     sem_t sem_;
+    struct timespec ts_;
+    bool sleepFallback_;
 
     static void StreamReadCallback_(pa_stream *s, size_t length, void *userdata);
     static void StreamWriteCallback_(pa_stream *s, size_t length, void *userdata);
@@ -101,6 +100,8 @@ private:
 #if 0
     static void StreamLatencyCallback_(pa_stream *p, void *userdata);
 #endif // 0
+
+    static void HandleXCPU_(int signum, siginfo_t *info, void *extra);
 };
 
 #endif // PULSE_AUDIO_DEVICE_H

src/gui/controls/CMakeLists.txt

@@ -3,7 +3,8 @@ add_library(fdv_gui_controls STATIC
     plot_scalar.cpp
     plot_scatter.cpp
     plot_spectrum.cpp
-    plot_waterfall.cpp)
+    plot_waterfall.cpp
+    ReportMessageRenderer.cpp)
 
 target_include_directories(fdv_gui_controls PRIVATE ${CODEC2_INCLUDE_DIRS} ${CMAKE_CURRENT_SOURCE_DIR}/../.. ${CMAKE_CURRENT_BINARY_DIR}/../..)
 

src/gui/controls/ReportMessageRenderer.cpp

@@ -0,0 +1,77 @@
+//==========================================================================
+// Name:            ReportMessageRenderer.cpp
+// Purpose:         Renderer for wxDataViewCtrl that helps data render properly
+//                  on all platforms.
+// Created:         April 14, 2024
+// Authors:         Mooneer Salem
+// 
+// License:
+//
+//  This program is free software; you can redistribute it and/or modify
+//  it under the terms of the GNU General Public License version 2.1,
+//  as published by the Free Software Foundation.  This program is
+//  distributed in the hope that it will be useful, but WITHOUT ANY
+//  WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//  FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public
+//  License for more details.
+//
+//  You should have received a copy of the GNU General Public License
+//  along with this program; if not, see <http://www.gnu.org/licenses/>.
+//
+//==========================================================================
+
+#include "ReportMessageRenderer.h"
+
+#include <wx/graphics.h>
+#include <wx/dc.h>
+
+ReportMessageRenderer::ReportMessageRenderer()
+    : wxDataViewCustomRenderer("string", wxDATAVIEW_CELL_INERT, wxALIGN_LEFT) { }
+
+bool ReportMessageRenderer::Render(wxRect cell, wxDC *dc, int state)
+{
+#if defined(WIN32)
+    wxGraphicsRenderer* renderer = wxGraphicsRenderer::GetDirect2DRenderer();
+    if (renderer != nullptr)
+    {
+        wxGraphicsContext* context = renderer->CreateContextFromUnknownDC(*dc);
+        if (context != nullptr)
+        {
+            wxColour color = dc->GetTextForeground();
+            if (state & wxDATAVIEW_CELL_SELECTED)
+            {
+                color = wxSystemSettings::GetColour(wxSYS_COLOUR_LISTBOXTEXT);
+            }
+            const wxString paintText =
+                wxControl::Ellipsize(m_value, *dc, GetEllipsizeMode(),
+                                     cell.GetWidth(), wxELLIPSIZE_FLAGS_NONE);
+            context->SetFont(dc->GetFont(), color);
+            context->DrawText(paintText, cell.x, cell.y);
+            delete context;
+            return true;
+        }
+    }
+#endif // defined(WIN32)
+
+    RenderBackground(dc, cell);
+    RenderText(m_value, 0, cell, dc, state);
+    
+    return true;
+}
+
+bool ReportMessageRenderer::SetValue( const wxVariant &value )
+{
+    m_value = value.GetString();
+    return true;
+}
+
+bool ReportMessageRenderer::GetValue( wxVariant &WXUNUSED(value) ) const 
+{ 
+    return true; 
+}
+
+wxSize ReportMessageRenderer::GetSize() const 
+{
+    // Basically just renders text.
+    return GetTextExtent(m_value);
+}

src/gui/controls/ReportMessageRenderer.h

@@ -0,0 +1,44 @@
+//==========================================================================
+// Name:            ReportMessageRenderer.h
+// Purpose:         Renderer for wxDataViewCtrl that helps data render properly
+//                  on all platforms.
+// Created:         April 14, 2024
+// Authors:         Mooneer Salem
+// 
+// License:
+//
+//  This program is free software; you can redistribute it and/or modify
+//  it under the terms of the GNU General Public License version 2.1,
+//  as published by the Free Software Foundation.  This program is
+//  distributed in the hope that it will be useful, but WITHOUT ANY
+//  WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//  FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public
+//  License for more details.
+//
+//  You should have received a copy of the GNU General Public License
+//  along with this program; if not, see <http://www.gnu.org/licenses/>.
+//
+//==========================================================================
+
+#ifndef REPORT_MESSAGE_RENDERER_H
+#define REPORT_MESSAGE_RENDERER_H
+
+#include <wx/dataview.h>
+
+class ReportMessageRenderer : public wxDataViewCustomRenderer
+{
+public:
+    ReportMessageRenderer();
+    virtual ~ReportMessageRenderer() = default;
+    
+    // Overrides from wxDataViewCustomRenderer
+    virtual bool Render (wxRect cell, wxDC *dc, int state) override;
+    virtual bool SetValue( const wxVariant &value ) override;
+    virtual bool GetValue( wxVariant &WXUNUSED(value) ) const override;
+    virtual wxSize GetSize() const override;
+
+private:
+    wxString m_value;
+};
+
+#endif // REPORT_MESSAGE_RENDERER_H

src/gui/dialogs/freedv_reporter.h

@@ -25,13 +25,15 @@
 #include <string>
 #include <map>
 #include <vector>
+#include <mutex>
 
-#include <wx/imaglist.h>
 #include <wx/tipwin.h>
+#include <wx/dataview.h>
 
 #include "main.h"
 #include "defines.h"
 #include "reporting/FreeDVReporter.h"
+#include "../controls/ReportMessageRenderer.h"
 
 //-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--=-=-=-=
 // Class FreeDVReporterDialog
@@ -71,6 +73,8 @@ class FreeDVReporterDialog : public wxFrame
         
         bool isTextMessageFieldInFocus();
     
+        void Unselect(wxDataViewItem& dvi) { m_listSpots->Unselect(dvi); }
+    
     protected:
 
         // Handlers for events.
@@ -94,24 +98,24 @@ class FreeDVReporterDialog : public wxFrame
         void    OnStatusTextChange(wxCommandEvent& event);
         void    OnSystemColorChanged(wxSysColourChangedEvent& event);
 
-        void OnItemSelected(wxListEvent& event);
+        void OnItemSelectionChanged(wxDataViewEvent& event);
-        void OnItemDeselected(wxListEvent& event);
+        void OnColumnClick(wxDataViewEvent& event);
-        void OnSortColumn(wxListEvent& event);
+        void OnItemDoubleClick(wxDataViewEvent& event);
+        void OnItemRightClick(wxDataViewEvent& event);
+
         void OnTimer(wxTimerEvent& event);
+        void DeselectItem();
+        void DeselectItem(wxMouseEvent& event);
         void AdjustToolTip(wxMouseEvent& event);
         void OnFilterTrackingEnable(wxCommandEvent& event);
-        void OnRightClickSpotsList(wxContextMenuEvent& event);
         void OnCopyUserMessage(wxCommandEvent& event);
         void SkipMouseEvent(wxMouseEvent& event);
         void AdjustMsgColWidth(wxListEvent& event);
-        
+        void OnRightClickSpotsList(wxContextMenuEvent& event);
-        void OnDoubleClick(wxMouseEvent& event);
                 
         // Main list box that shows spots
-        wxListView*   m_listSpots;
+        wxDataViewCtrl*   m_listSpots;
-        wxImageList*  m_sortIcons;
+        wxObjectDataPtr<wxDataViewModel> spotsDataModel_;
-        int upIconIndex_;
-        int downIconIndex_;
         wxMenu* spotsPopupMenu_;
         wxString tempUserMessage_; // to store the currently hovering message prior to going on the clipboard
 
@@ -139,86 +143,153 @@ class FreeDVReporterDialog : public wxFrame
         // Timer to unhighlight RX rows after 10s (like with web-based Reporter)
         wxTimer* m_highlightClearTimer;
 
-        std::vector<std::function<void()> > fnQueue_;
-        std::mutex fnQueueMtx_;
-
         wxTipWindow* tipWindow_;
         
      private:
-         struct ReporterData
+         class FreeDVReporterDataModel : public wxDataViewModel
          {
-             std::string sid;
+         public:
-             wxString callsign;
+             FreeDVReporterDataModel(FreeDVReporterDialog* parent);
-             wxString gridSquare;
+             virtual ~FreeDVReporterDataModel();
-             double distanceVal;
-             wxString distance;
-             double headingVal;
-             wxString heading;
-             wxString version;
-             uint64_t frequency;
-             wxString freqString;
-             wxString status;
-             wxString txMode;
-             bool transmitting;
-             wxString lastTx;
-             wxDateTime lastTxDate;
-             wxDateTime lastRxDate;
-             wxString lastRxCallsign;
-             wxString lastRxMode;
-             wxString snr;
-             wxString lastUpdate;
-             wxDateTime lastUpdateDate;
-             wxString userMessage;
-             wxDateTime lastUpdateUserMessage;
-         };
 
-         std::shared_ptr<FreeDVReporter> reporter_;
+             void setReporter(std::shared_ptr<FreeDVReporter> reporter);
-         std::map<int, int> columnLengths_;
+             void setBandFilter(FilterFrequency freq);
-         std::map<std::string, ReporterData*> allReporterData_;
+             void refreshAllRows();
-         FilterFrequency currentBandFilter_;
+             void requestQSY(wxDataViewItem selectedItem, uint64_t frequency, wxString customText);
-         int currentSortColumn_;
+             void updateHighlights();
-         bool sortAscending_;
+             void updateMessage(wxString statusMsg)
-         bool isConnected_;
+             {
-         bool filterSelfMessageUpdates_;
+                 if (reporter_)
-         uint64_t filteredFrequency_;
+                 {
+                     reporter_->updateMessage(statusMsg.utf8_string());
+                 }
+             }
 
-         void clearAllEntries_(bool clearForAllBands);
+             uint64_t getFrequency(wxDataViewItem item)
-         void onReporterConnect_();
+             {
-         void onReporterDisconnect_();
+                if (item.IsOk())
-         void onUserConnectFn_(std::string sid, std::string lastUpdate, std::string callsign, std::string gridSquare, std::string version, bool rxOnly);
+                {
-         void onUserDisconnectFn_(std::string sid, std::string lastUpdate, std::string callsign, std::string gridSquare, std::string version, bool rxOnly);
+                    auto data = (ReporterData*)item.GetID();
-         void onFrequencyChangeFn_(std::string sid, std::string lastUpdate, std::string callsign, std::string gridSquare, uint64_t frequencyHz);
+                    return data->frequency;
-         void onTransmitUpdateFn_(std::string sid, std::string lastUpdate, std::string callsign, std::string gridSquare, std::string txMode, bool transmitting, std::string lastTxDate);
+                }
-         void onReceiveUpdateFn_(std::string sid, std::string lastUpdate, std::string callsign, std::string gridSquare, std::string receivedCallsign, float snr, std::string rxMode);
-         void onMessageUpdateFn_(std::string sid, std::string lastUpdate, std::string message);
-         void onConnectionSuccessfulFn_();
-         void onAboutToShowSelfFn_();
 
-         wxString makeValidTime_(std::string timeStr, wxDateTime& timeObj);
+                return 0;
+             }
 
-         void addOrUpdateListIfNotFiltered_(ReporterData* data, std::map<int, int>& colResizeList);
+             FilterFrequency getCurrentBandFilter() const { return currentBandFilter_; }
-         FilterFrequency getFilterForFrequency_(uint64_t freq);
+             wxString getCallsign(wxDataViewItem& item)
-         bool isFiltered_(uint64_t freq);
+             {
+                 if (item.IsOk())
+                 {
+                     auto data = (ReporterData*)item.GetID();
+                     return data->callsign;
+                 }
+                 return "";
+             }
              
-         bool setColumnForRow_(int row, int col, wxString val, std::map<int, int>& colResizeList);
+             wxString getUserMessage(wxDataViewItem& item)
-         void resizeChangedColumns_(std::map<int, int>& colResizeList);
+             {
+                 if (item.IsOk())
+                 {
+                     auto data = (ReporterData*)item.GetID();
+                     return data->userMessage;
+                 }
+                 return "";
+             }
              
-         void sortColumn_(int col);
+             bool isValidForReporting()
-         void sortColumn_(int col, bool direction);
+             {
+                 return reporter_ && reporter_->isValidForReporting();
+             }
 
-         double calculateDistance_(wxString gridSquare1, wxString gridSquare2);
+             // Required overrides to implement functionality
-         double calculateBearingInDegrees_(wxString gridSquare1, wxString gridSquare2);
+             virtual bool HasDefaultCompare() const override;
-         void calculateLatLonFromGridSquare_(wxString gridSquare, double& lat, double& lon);
+             virtual int Compare (const wxDataViewItem &item1, const wxDataViewItem &item2, unsigned int column, bool ascending) const override;
+             virtual bool GetAttr (const wxDataViewItem &item, unsigned int col, wxDataViewItemAttr &attr) const override;
+             virtual unsigned int GetChildren (const wxDataViewItem &item, wxDataViewItemArray &children) const override;
+             virtual wxDataViewItem GetParent (const wxDataViewItem &item) const override;
+             virtual void GetValue (wxVariant &variant, const wxDataViewItem &item, unsigned int col) const override;
+             virtual bool IsContainer (const wxDataViewItem &item) const override;
+             virtual bool SetValue (const wxVariant &variant, const wxDataViewItem &item, unsigned int col) override;
 
-         void execQueuedAction_();
+         private:
+             struct ReporterData
+             {
+                std::string sid;
+                wxString callsign;
+                wxString gridSquare;
+                double distanceVal;
+                wxString distance;
+                double headingVal;
+                wxString heading;
+                wxString version;
+                uint64_t frequency;
+                wxString freqString;
+                wxString status;
+                wxString txMode;
+                bool transmitting;
+                wxString lastTx;
+                wxDateTime lastTxDate;
+                wxDateTime lastRxDate;
+                wxString lastRxCallsign;
+                wxString lastRxMode;
+                wxString snr;
+                wxString lastUpdate;
+                wxDateTime lastUpdateDate;
+                wxString userMessage;
+                wxDateTime lastUpdateUserMessage;
+                wxDateTime connectTime;
 
-         void resizeAllColumns_();
+                // Controls whether this row has been filtered
-         int getSizeForTableCellString_(wxString string);
+                bool isVisible;
 
-         static wxCALLBACK int ListCompareFn_(wxIntPtr item1, wxIntPtr item2, wxIntPtr sortData);
+                // Controls the current highlight color
-         static double DegreesToRadians_(double degrees);
+                wxColour foregroundColor;
-         static double RadiansToDegrees_(double radians);
+                wxColour backgroundColor;
-         static wxString GetCardinalDirection_(int degrees);
+            };
+
+            std::shared_ptr<FreeDVReporter> reporter_;
+            std::map<std::string, ReporterData*> allReporterData_;
+            std::vector<std::function<void()> > fnQueue_;
+            std::mutex fnQueueMtx_;
+            std::recursive_mutex dataMtx_;
+            bool isConnected_;
+            FreeDVReporterDialog* parent_;
+
+            FilterFrequency currentBandFilter_;
+            bool filterSelfMessageUpdates_;
+            uint64_t filteredFrequency_;
+
+            bool isFiltered_(uint64_t freq);
+
+            void clearAllEntries_();
+
+            void onReporterConnect_();
+            void onReporterDisconnect_();
+            void onUserConnectFn_(std::string sid, std::string lastUpdate, std::string callsign, std::string gridSquare, std::string version, bool rxOnly);
+            void onUserDisconnectFn_(std::string sid, std::string lastUpdate, std::string callsign, std::string gridSquare, std::string version, bool rxOnly);
+            void onFrequencyChangeFn_(std::string sid, std::string lastUpdate, std::string callsign, std::string gridSquare, uint64_t frequencyHz);
+            void onTransmitUpdateFn_(std::string sid, std::string lastUpdate, std::string callsign, std::string gridSquare, std::string txMode, bool transmitting, std::string lastTxDate);
+            void onReceiveUpdateFn_(std::string sid, std::string lastUpdate, std::string callsign, std::string gridSquare, std::string receivedCallsign, float snr, std::string rxMode);
+            void onMessageUpdateFn_(std::string sid, std::string lastUpdate, std::string message);
+
+            void onConnectionSuccessfulFn_();
+            void onAboutToShowSelfFn_();
+
+            void execQueuedAction_();
+
+            wxString makeValidTime_(std::string timeStr, wxDateTime& timeObj);
+            double calculateDistance_(wxString gridSquare1, wxString gridSquare2);
+            double calculateBearingInDegrees_(wxString gridSquare1, wxString gridSquare2);
+            void calculateLatLonFromGridSquare_(wxString gridSquare, double& lat, double& lon);
+
+            static double DegreesToRadians_(double degrees);
+            static double RadiansToDegrees_(double radians);
+            static wxString GetCardinalDirection_(int degrees);
+        };
+
+        FilterFrequency getFilterForFrequency_(uint64_t freq);
+        bool sortRequired_;
 };
 
 #endif // __FREEDV_REPORTER_DIALOG__

src/main.cpp

@@ -484,19 +484,19 @@ bool MainApp::OnCmdLineParsed(wxCmdLineParser& parser)
         {
             log_info("Will transmit for %d seconds", utTxTimeSeconds);
         }
-	else
+        else
-	{
+        {
             utTxTimeSeconds = 60;
-	}
+        }
 
         if (parser.Found("txattempts", (long*)&utTxAttempts))
         {
             log_info("Will transmit %d time(s)", utTxAttempts);
         }
-	else
+        else
-	{
+        {
             utTxAttempts = 1;
-	}
+        }
     }
     
     if (parser.Found("rxfeaturefile", &utRxFeatureFile))
@@ -1002,7 +1002,7 @@ setDefaultMode:
     }
 
     // Initialize FreeDV Reporter as required
-    initializeFreeDVReporter_();
+    CallAfter([&]() { initializeFreeDVReporter_(); });
     
     // If the FreeDV Reporter window was open on last execution, reopen it now.
     CallAfter([&]() {
@@ -1106,17 +1106,17 @@ MainFrame::MainFrame(wxWindow *parent) : TopFrame(parent, wxID_ANY, _("FreeDV ")
     // Add Demod Input window
     m_panelDemodIn = new PlotScalar((wxFrame*) m_auiNbookCtrl, 1, WAVEFORM_PLOT_TIME, 1.0/WAVEFORM_PLOT_FS, -1, 1, 1, 0.2, "%2.1f", 0);
     m_auiNbookCtrl->AddPage(m_panelDemodIn, _("Frm Radio"), true, wxNullBitmap);
-    g_plotDemodInFifo = codec2_fifo_create(4*WAVEFORM_PLOT_BUF);
+    g_plotDemodInFifo = codec2_fifo_create(10*WAVEFORM_PLOT_FS);
 
     // Add Speech Input window
     m_panelSpeechIn = new PlotScalar((wxFrame*) m_auiNbookCtrl, 1, WAVEFORM_PLOT_TIME, 1.0/WAVEFORM_PLOT_FS, -1, 1, 1, 0.2, "%2.1f", 0);
     m_auiNbookCtrl->AddPage(m_panelSpeechIn, _("Frm Mic"), true, wxNullBitmap);
-    g_plotSpeechInFifo = codec2_fifo_create(4*WAVEFORM_PLOT_BUF);
+    g_plotSpeechInFifo = codec2_fifo_create(10*WAVEFORM_PLOT_FS);
 
     // Add Speech Output window
     m_panelSpeechOut = new PlotScalar((wxFrame*) m_auiNbookCtrl, 1, WAVEFORM_PLOT_TIME, 1.0/WAVEFORM_PLOT_FS, -1, 1, 1, 0.2, "%2.1f", 0);
     m_auiNbookCtrl->AddPage(m_panelSpeechOut, _("To Spkr/Hdphns"), true, wxNullBitmap);
-    g_plotSpeechOutFifo = codec2_fifo_create(4*WAVEFORM_PLOT_BUF);
+    g_plotSpeechOutFifo = codec2_fifo_create(10*WAVEFORM_PLOT_FS);
 
     // Add Timing Offset window
     m_panelTimeOffset = new PlotScalar((wxFrame*) m_auiNbookCtrl, 1, 5.0, DT, -0.5, 0.5, 1, 0.1, "%2.1f", 0);
@@ -1703,7 +1703,7 @@ void MainFrame::OnTimer(wxTimerEvent &evt)
         float snr_limited;
         // some APIs pass us invalid values, so lets trap it rather than bombing
         float snrEstimate = freedvInterface.getSNREstimate();
-        if (!(isnan(snrEstimate) || isinf(snrEstimate))) {
+        if (!(isnan(snrEstimate) || isinf(snrEstimate)) && freedvInterface.getSync()) {
             g_snr = m_snrBeta*g_snr + (1.0 - m_snrBeta)*snrEstimate;
         }
         snr_limited = g_snr;
@@ -1814,7 +1814,7 @@ void MainFrame::OnTimer(wxTimerEvent &evt)
             if (oldColor != newColor)
             {
                 m_textSync->SetForegroundColour(newColor);
-        	    m_textSync->SetLabel("Modem");
+                    m_textSync->SetLabel("Modem");
                 m_textSync->Refresh();
             }
         }
@@ -3125,9 +3125,16 @@ void MainFrame::startRxStream()
         // (depending on platform/audio library). Sample rate conversion, 
         // stats for spectral plots, and transmit processng are all performed 
         // in the tx/rxProcessing loop.
-
+        //
+        // Note that soundCard1InFifoSizeSamples is significantly larger than
+        // the other FIFO sizes. This is to better handle PulseAudio/pipewire
+        // behavior on some devices, where the system sends multiple *seconds*
+        // of audio samples at once followed by long periods with no samples at
+        // all. Without a very large FIFO size (or a way to dynamically change
+        // FIFO sizes, which isn't recommended for real-time operation), we will
+        // definitely lose audio.
         int m_fifoSize_ms = wxGetApp().appConfiguration.fifoSizeMs;
-        int soundCard1InFifoSizeSamples = m_fifoSize_ms*wxGetApp().appConfiguration.audioConfiguration.soundCard1In.sampleRate / 1000;
+        int soundCard1InFifoSizeSamples = 10 * wxGetApp().appConfiguration.audioConfiguration.soundCard1In.sampleRate;
         int soundCard1OutFifoSizeSamples = m_fifoSize_ms*wxGetApp().appConfiguration.audioConfiguration.soundCard1Out.sampleRate / 1000;
 
         if (txInSoundDevice && txOutSoundDevice)
@@ -3266,26 +3273,22 @@ void MainFrame::startRxStream()
             rxOutSoundDevice->setOnAudioData([](IAudioDevice& dev, void* data, size_t size, void* state) {
                 paCallBackData* cbData = static_cast<paCallBackData*>(state);
                 short* audioData = static_cast<short*>(data);
-                short* outdata = new short[size];
+                short outdata = 0;
-                assert(outdata != nullptr);
 
-                int result = codec2_fifo_read(cbData->outfifo2, outdata, size);
+                if ((size_t)codec2_fifo_used(cbData->outfifo2) < size)
-                if (result == 0) 
-                {
-                    for (size_t i = 0; i < size; i++)
-                    {
-                        for (int j = 0; j < dev.getNumChannels(); j++)
-                        {
-                            *audioData++ = outdata[i];
-                        }
-                    }
-                }
-                else 
                 {
                     g_outfifo2_empty++;
+                    return;
                 }
 
-                delete[] outdata;
+                for (; size > 0; size--)
+                {
+                    codec2_fifo_read(cbData->outfifo2, &outdata, 1);
+                    for (int j = 0; j < dev.getNumChannels(); j++)
+                    {
+                        *audioData++ = outdata;
+                    }
+                }
             }, g_rxUserdata);
             
             rxOutSoundDevice->setOnAudioOverflow([](IAudioDevice& dev, void* state)
@@ -3329,47 +3332,34 @@ void MainFrame::startRxStream()
             txOutSoundDevice->setOnAudioData([](IAudioDevice& dev, void* data, size_t size, void* state) {
                 paCallBackData* cbData = static_cast<paCallBackData*>(state);
                 short* audioData = static_cast<short*>(data);
-                short* outdata = new short[size];
+                short outdata = 0;
-                assert(outdata != nullptr);
                
-                unsigned int available = std::min(codec2_fifo_used(cbData->outfifo1), (int)size);
+                if ((size_t)codec2_fifo_used(cbData->outfifo1) < size)
-
-                int result = codec2_fifo_read(cbData->outfifo1, outdata, available);
-                if (result == 0) 
                 {
+                    g_outfifo1_empty++;
+                    return;
+                }
+
+                for (; size > 0; size--, audioData += dev.getNumChannels())
+                {
+                    codec2_fifo_read(cbData->outfifo1, &outdata, 1);
+
                     // write signal to all channels to start. This is so that
                     // the compiler can optimize for the most common case.
-                    for(size_t i = 0; i < available; i++, audioData += dev.getNumChannels()) 
+                    for (auto j = 0; j < dev.getNumChannels(); j++)
                     {
-                        for (auto j = 0; j < dev.getNumChannels(); j++)
+                        audioData[j] = outdata;
-                        {
-                            audioData[j] = outdata[i];
-                        }
                     }
                     
                     // If VOX tone is enabled, go back through and add the VOX tone
                     // on the left channel.
                     if (cbData->leftChannelVoxTone)
                     {
-                        for(size_t i = 0; i < size; i++, audioData += dev.getNumChannels())
+                        cbData->voxTonePhase += 2.0*M_PI*VOX_TONE_FREQ/wxGetApp().appConfiguration.audioConfiguration.soundCard1Out.sampleRate;
-                        {
+                        cbData->voxTonePhase -= 2.0*M_PI*floor(cbData->voxTonePhase/(2.0*M_PI));
-                            cbData->voxTonePhase += 2.0*M_PI*VOX_TONE_FREQ/wxGetApp().appConfiguration.audioConfiguration.soundCard1Out.sampleRate;
+                        audioData[0] = VOX_TONE_AMP*cos(cbData->voxTonePhase);
-                            cbData->voxTonePhase -= 2.0*M_PI*floor(cbData->voxTonePhase/(2.0*M_PI));
-                            audioData[0] = VOX_TONE_AMP*cos(cbData->voxTonePhase);
-                        }
-                    }
-                    
-                    if (size != available)
-                    {
-                        g_outfifo1_empty++;
                     }
                 }
-                else 
-                {
-                    g_outfifo1_empty++;
-                }
-
-                delete[] outdata;
             }, g_rxUserdata);
         
             txOutSoundDevice->setOnAudioOverflow([](IAudioDevice& dev, void* state)
@@ -3390,27 +3380,22 @@ void MainFrame::startRxStream()
             rxOutSoundDevice->setOnAudioData([](IAudioDevice& dev, void* data, size_t size, void* state) {
                 paCallBackData* cbData = static_cast<paCallBackData*>(state);
                 short* audioData = static_cast<short*>(data);
+                short outdata = 0;
 
-                short* outdata = new short[size];
+                if ((size_t)codec2_fifo_used(cbData->outfifo1) < size)
-                assert(outdata != nullptr);
-
-                int result = codec2_fifo_read(cbData->outfifo1, outdata, size);
-                if (result == 0) 
-                {
-                    for (size_t i = 0; i < size; i++)
-                    {
-                        for (int j = 0; j < dev.getNumChannels(); j++)
-                        {
-                            *audioData++ = outdata[i];
-                        }
-                    }
-                }
-                else 
                 {
                     g_outfifo1_empty++;
+                    return;
                 }
 
-                delete[] outdata;
+                for (; size > 0; size--)
+                {
+                    codec2_fifo_read(cbData->outfifo1, &outdata, 1);
+                    for (int j = 0; j < dev.getNumChannels(); j++)
+                    {
+                        *audioData++ = outdata;
+                    }
+                }
             }, g_rxUserdata);
             
             rxOutSoundDevice->setOnAudioOverflow([](IAudioDevice& dev, void* state)

src/ongui.cpp

@@ -858,6 +858,8 @@ void MainFrame::togglePTT(void) {
         {
             log_info("Waiting for EOO to be queued");
             endingTx = true;
+            
+            auto beginTime = std::chrono::high_resolution_clock::now();
             while(true)
             {
                 if (g_eoo_enqueued)
@@ -868,6 +870,13 @@ void MainFrame::togglePTT(void) {
  
                 wxThread::Sleep(1);
                 wxGetApp().Yield(true);
+
+                auto endTime = std::chrono::high_resolution_clock::now();
+                if ((endTime - beginTime) >= std::chrono::seconds(2))
+                {
+                    log_warn("Timed out waiting for EOO to be enqueued");
+                    break;
+                }
             }
         }
 

src/pipeline/TapStep.cpp

@@ -23,6 +23,7 @@
 #include "TapStep.h"
 
 #include <assert.h>
+#include <future>
 
 TapStep::TapStep(int sampleRate, IPipelineStep* tapStep, bool operateBackground)
     : tapStep_(tapStep)
@@ -34,7 +35,15 @@ TapStep::TapStep(int sampleRate, IPipelineStep* tapStep, bool operateBackground)
 
 TapStep::~TapStep()
 {
-    // empty
+    // Make sure we clear everything remaining in queue before
+    // deallocating. This isn't done in the base class as tapStep_
+    // could be deallocated by the time we call that class' destructor.
+    auto prom = std::make_shared<std::promise<void>>();
+    auto fut = prom->get_future();
+    enqueue_([&]() {
+        prom->set_value();
+    });
+    fut.wait();
 }
 
 int TapStep::getInputSampleRate() const

src/pipeline/TxRxThread.cpp

@@ -624,7 +624,7 @@ void TxRxThread::txProcessing_()
         int             nout;
 
         
-        while((unsigned)codec2_fifo_free(cbData->outfifo1) >= nsam_one_modem_frame) {        
+        while(!helper_->mustStopWork() && (unsigned)codec2_fifo_free(cbData->outfifo1) >= nsam_one_modem_frame) {        
             // OK to generate a frame of modem output samples we need
             // an input frame of speech samples from the microphone.
 
@@ -743,14 +743,15 @@ void TxRxThread::rxProcessing_()
         clearFifos_();
     }
 
-    // while we have enough input samples available ... 
+    int nsam_one_speech_frame = freedvInterface.getRxNumSpeechSamples() * ((float)outputSampleRate_ / (float)freedvInterface.getRxSpeechSampleRate());
-    while (codec2_fifo_read(cbData->infifo1, inputSamples_.get(), nsam) == 0 && processInputFifo) {
+    auto outFifo = (g_nSoundCards == 1) ? cbData->outfifo1 : cbData->outfifo2;
 
+    // while we have enough input samples available and enough space in the output FIFO ... 
+    while (!helper_->mustStopWork() && codec2_fifo_free(outFifo) >= nsam_one_speech_frame && codec2_fifo_read(cbData->infifo1, inputSamples_.get(), nsam) == 0 && processInputFifo) {
         // send latest squelch level to FreeDV API, as it handles squelch internally
         freedvInterface.setSquelch(g_SquelchActive, g_SquelchLevel);
 
         auto outputSamples = pipeline_->execute(inputSamples_, nsam, &nout);
-        auto outFifo = (g_nSoundCards == 1) ? cbData->outfifo1 : cbData->outfifo2;
         
         if (nout > 0 && outputSamples.get() != nullptr)
         {

src/util/IRealtimeHelper.h

@@ -41,6 +41,10 @@ public:
     
     // Reverts real-time priority for current thread.
     virtual void clearHelperRealTime() = 0;
+
+    // Returns true if real-time thread MUST sleep ASAP. Failure to do so
+    // may result in SIGKILL being sent to the process by the kernel.
+    virtual bool mustStopWork() = 0;
 };
 
 #endif