Resolved MaximDL Settings for STT-8300 with self guiding filter wheel

For Camera 1, [Setup Camera], select Camera Model = SBIG Universal in the drop down..
Then select Guide Chip drop down = External.
This tells it to use the External guide sensor (eg remote guide head), which is the port on the camera that connects to the guide sensor in the FW8G-STT.

Then Camera 1's filter wheel click [Setup Filter]
Filter or Controlling Camera Model = SBIG Universal
Model = Standard
COM Port = don't need to specify; COM1 will be ignored.

Then in Camera 1 Turn on [x] Dual Chip Mode.
Dual chip will grey out the Camera 2; it automatically knows about the main camera having the guide sensor.

 

Check SBIG Driver Checker is 4.05, and make sure sbigudrv.dll is 5.1.1.0.
Then check the camera firmware is current, using the firmware tab in the driver checker.

 

Here's a direct link to save you time:
https://diffractionlimited.com/downloads/SetupDriverChecker64.exe

 

You do have the HDMI cable between the FW8G-STT and the STT camera body, in addition to the I2C AUX cable for the filter wheel side?

 

Could you send the FITS image from the guide sensor? It might give us a hint.
Could be the guide hardware has a problem.

 

Good! I was about to suggest a new HDMI cable.
Also, in MaxIm, File... Settings, Site and Optics. Fill in all the parameters.
The guider has a 0.7x focal reducer, so when you put in the guider optical parameters, it is the same as the main scope, EXCEPT for the focal length x 0.7.