Main feature

Computer flight simulator (MSFS2020 or Flight Gear) running in "visualization only mode" (simulation of everything is turned off, telemetry and controls, that are going through the machine where simulator is running on, are binded to the gauges/ control surfaces of the model. GPS coordinates are matched. Scenery transparency can be tuned, behind it - pan tilt servo coordinated (or 360) video feed from analog source. If video feed is lost, scenery becomes solid.

Future development

Ona aircraft FPGA video filter

that deals with camera, and passes it, adding hdmi output from raspberry pi

NeTV2 fits the bill. May be better to opt in for digital camera.

Risks

Consider usefulness of 2 same system running in parallel

If one of them (on the ground) looses telemetry, and updated telemetry is wastly different, automatic course correction can be attempted?

MSFS2020 is poisonous idea. Reliance on it will totally murder this project.

Dont even consider existance of MSFS2020 before running prototype.

MSFS2020 wont support required APIs

Write to microsoft with prototype implementation

Why Raspberry Pi gotta go flying?

Whole thing could be done on the laptop as long as it takes and outputs analog video signal, and has ways to switch them

Real requirement is Flight Simulator (could be MSFS2020, hell why not) getting telemetry and video feed from the model + awareness of head tracking

Mavlink is the only option, ardupilot is basically requirement

Latency of video feed gonna be slow

We are going with servo implementation. There should be a control that switches videofeed between the desktop of raspberry pi running high latency stuff and "Backup Flight controller"

Ditch whole raspberry pi and and flight gear foolishness and just code whole thing in FPGA - stupid. every flight controller manufacturer did that. Not flexible. No future. Wont let people fly their Boeings in goggles with boeing cockpit and nav instruments

Quality of video feed gonna be inferior

no mitigation besides time and development of superior digital transmitters and better cameras?

Using off the shelf rc cameras + servos

Quality of video won't be enough for looking at instruments

Make RC specific cockpit in Flight Gear with large instruments

Wont be able to find proper servos

Movement of the camera will be misaligned with movement of flightgear cockpit

Input servo speeds as hardcoded values

Picam360 can be too slow. Best known latency is 120ms

Mitigation 1: API for selective refresh of pixel

requested from developers waiting for answer

Mitigation 2: Ditching Picam360, using 2 servos and regular cheap picam

will need to syncronize view then. although if we got such view syncronization then it can lower the cost of the kit - $130.

Real ADF/VOR receivers are unavailable

Who cares we simulate them

Make our own, no certification needed, its just a receiver

FAA wont calculate IFR hours on our machines for real ones

Who cares maybe it will

Aircraft

ARF frame

Any frame with suitable cockpit is good. 1450mm wingspan is good measure - it will fit raspberry with all extensions nicely plane1 plane2

RC Receiver

VTX

Picam360

Raspberry PI 4

Navio 2 Flight controller RPi shield

Ground Station

Raspberry PI 4

Touch display for RPi

Any USB Joystick

RC Transmitter

Mavlink

Antenna tracker servos

Video receiver/Goggles/head tracking

Software

Aircraft

FlightGear

Scenery is off

Gauges/control surfaces of model are syncronized with onboard Navio2 flight controller.

"Here is ground" overlay based on altimeter/aircraft attitude

Picam360 view is drawn instead of scenery

Picam360 receives api call (to be implemented by manufacturer) to enable selective (only in field of view) update of pixels

"Here is ground" overlay based on height data preloaded into raspberry pi.

Aerial restriction zones overlay from aerospace navigational databases

Make sure Flight Gear's in cockpit GPS works as expected

Reroute control surfaces through flightgear properly: control surfaces are inputs of the joystick, flight gear decides what to do with them

Make sure Flight Gear's autopilot system works with Navio2 and real aircraft

Improve Flight Gear's autopilot system to be more useful for rc aircraft

In-headset configuration menu if needed

Fake ADF and VOR receivers based on attitude/gps/navigational data of flight gear

Real ADF/VOR receiver, connect to Flight Gear cockpit

RPi runs in wifi access point mode

On connection to this wifi access point webpage opens with the pilot's point of view (simple desktop stream). Chat visible to pilot is available

Ground station

Mavlink ground station software

should run the antenna tracker servos

USB joystick to ppm conventor

https://github.com/jsa/flystick

Configurator/calibrator of rates/curves etc in the touch interface

^^^ this could be replaced by either Flight Gear running as joystick input preprocessor, or with OpenTx companion radio simulator outputing ppm. Need to check flight gear's joystick config interface

Ground Flight Gear mavlink sync of state

Flight Gear on the ground will have the same state as Flight Gear on the aircraft

Detection of video data loss, switching feed to the ground Flight Gear, return of the remote feed view when feed is restored