So, this is my second attempt at the same goal: Build a 320x240 LED board and then run some software on a Pi that lets people interact with the thing in fun ways.
My original idea was: Connect two cameras to the Pi, one regular and one thermal cam, then combine the two images where the thermal’s heat index affects the saturation of the main cam image. The hope was that this would make people standing in front of the cams to “glow” where they’re warmer. This was to be used at an event at night or late evening time, where people are lightly dressed and possibly on mushrooms 
Now, I managed to build the board last summer, with 4 rows with 6 panels each, each row driven by a Raspi Pico W (Pimoroni Interstate 75), and a controlling Pi 5 that would send packts for each row to the Picos over WiFi UDP. That worked quite fine, though I could only get about 10 fps out of it. Then my thermal cam broke and I only had a regular cam, which wasn’t that great.
Now I wanted to go at it again, and still have not replaced the thermal cam, but I found this project which makes driving the matrix much easier, at least, and at a higher FPS.
So, without the ability to realize my original idea, and having 3 more days to get something done for the next event coming weekend, I like to try something else.
And that’s why I could use your input to see what you’ve made or think possible:
A few of my ideas (based on using a Pi 4 B to drive the matrix):
-
I have two Playstation 5 controllers. I guess I could connect them to the Pi 4 and then run some old school games on it. But which games? I have not run any games or emulators on a Pi before, so instead of my spending hours trying various things, I wonder if you have some things that you know to work and that are not too much of a hassle to install?
-
A generic graphics display that takes sound input. So, basically a funky “laser” show on the matrix. Which software would I use for that?
-
Using the Pi Model 3 12 MP cam and modify the image in funky ways for display on the matrix. What kind of effects would work that? Ideally, a “comicalize” operation would be cool, but a good one requires more computing power (i.e. a GPU), which the Pi can’t manage. Though, I might just use a Laptop (ideally, a new Macbook) for that task, and then send the generated frames to the Pi. The question here would be: How do I set up the Pi to receive the stream from the Mac over the network and send it to the matrix - is there already a program for that?
I’ve tried two different approaches to get retro games running on an LED matrix.
First, I used an HDMI capture setup. I ran an Atari emulator on a Pi and sent the HDMI output to another Pi-based LED matrix controller. This setup worked great because one Pi handles all the emulation heavy lifting while the other Pi just focuses on driving the display. Plus, you can display anything that outputs HDMI.
Second, I ran both the Atari emulator and the LED matrix controller directly on the same Pi. The emulator is still under development and doesn’t support audio yet, but key events work—so I can actually play the games.
Thanks for your input. I’m currently trying out RetroPie, and I can run some games on the HDMI.
But how did you get the 2nd Pi (that drives the matrix) to read the HDMI output of the first? Did you connect some video capture hardware? Over USB? And did you get it forwarded to the matrix, then?
For your second method, what did you - roughly - do to get the emulator’s output into the matrix?
The HDMI method was accomplished using OpenCV and an HDMI capture device like this one.
All you really need is a frame buffer to draw to your LED matrix.
For the HDMI way, I capture the image, process it, and use that to fill my frame buffer.
For the other method, I just grab the emulator’s frame buffer and hand it off to the matrix control function.
I’ve been looking for a way to access the emulator’s frame buffer. Do you use a specific emulator or are you using Retropie for it? Because I cannot find a way to access RetroPie’s output, yet. It looks complicated.
Sorry, I didn’t explain that clearly. You’ll need access to the emulator’s source code to extract the frame buffer. In my case, I wrote my own emulator for the project, so that part was straightforward.
I assume RetroPie publishes the source code? Also, there’s the Stella project that might be worth looking into.
1 Like
I am considering the dual-Pi setup with the HDMI connection. Does the Video Capture Card you used work like a standard USB video capture device, so that it’s just another frame grabber I need to run to read from it? It’s difficult to find information about that. I have, for instance, GameCapture devices from Elgato, and they appear to use a proprietary format.
I believe mine is a standard USB video capture device. From the Pi’s perspective, it shows up as a webcam. This page was really helpful during development. As it shows, the process is pretty straightforward. You just open the video capture, set the parameters, and start reading frames.