16803 FW7 not recognised... plus recent camera crashes

Discussion in 'STX and STXL Series Cameras' started by James Pierce, Mar 12, 2021.

  1. James Pierce

    James Pierce Cyanogen Customer

    Joined:
    May 20, 2019
    Messages:
    62
    Couple of recurring issues in the last few weeks I'm wondering if anyone can share some insight on...

    My system has been both stable and unchanged for a long time (6 months?)... since MaximDL 6.24 things have run very well. In the last few weeks (perhaps related to a windows update, not sure, they happen so often without choice!) I've found my 340 all sky camera has needed to have it's software restarted about every 3-4 days (again it has been stable for a very long time) and I've had quite a few nights cut short by MaximDL crashing or hanging and taking ACP with it.

    Now tonight my camera will not connect at all with an error related to the FW7.

    "Error (6) initating filter wheel"

    I'll attach the logs from the last ~5 nights so you can get a sense of it. I can manage to connect to the camera, but only if I remove the filter wheel from the config. The same is true via The SkyX, MaximDL and CCDOps.

    Things I have tired (but have not solved the problem):
    Power cycles of the camera and reboots of windows.
    Repeated windows updates.
    I've checked the usual Windows USB power management settings that sometimes get changed with updates and they are ok.
    I've updated to MaximDL 6.26

    I'm at a bit of a loss as to what to try from here.
     

    Attached Files:

  2. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    5,250
    Location:
    Earth
    Error 6 is a bad parameter - usually this is caused by not setting the filter wheel Model to Standard in the Setup Filter Wheel, SBIG AO w/FW screen. Like so:
    setup_filterwheel.png

    You mention MaxIm hanging. That's usually caused by an underlying hardware issue.
    Your log files are chopped off - are you killing MaxIm from the task manager?
    Several are missing the ending line:
    MaxIm DL Version 6.24 exited
    This can sometimes leave the SBIG universal driver and the hardware in a bad state.

    There's nothing here about the AllSk340M. I recall you had problems with it back in summer 2019, and I believe you resolved it back then. Perhaps we should tackle that separately.

    My approach would be similar to some of what you have already tried.
    1. Run Windows Update multiple times, and make sure to reboot after each install of an update. There may be something missing/modified/corrupted.
    2. Google "SFC /scannow" and "DISM" - this will explain how to use these command to clean up problems with your Windows installation.

    3. Verify everything listed in here - there's some subtleties to the Windows power management and USB that are easy to miss:
    https://forum.diffractionlimited.co...connects-turn-off-usb-selective-suspend.7848/
    Note the last item concerning USB hubs that Microsoft broke/fixed.

    4. I'd recommend checking the hardware using CCDOps. It's simpler.
    Take a bunch of exposures to exercise the filter wheel.
    Sometimes Error 6 is caused by a hardware failure.
    With the power off to the camera equipment, and USB disconnected, you may want to remove and reconnect the white pigtail cable from the filter wheel to the camera.
    Same for the AO-X.
    We might need to loop in @Bill in case there is a filter wheel failure of some kind.

    5. Provide the serial numbers of the FW and the rest of the gear. Just in case it gives us any insight.

    6. Event Viewer - Application Error
    If MaxIm hangs, sometimes we get a clue from the Windows progam called Event Viewer.
    Launch Event Viewer.
    Windows Logs
    Application
    Then look through the center window for the (X) Error - Application Error.
    We're looking for something like this:
    event_applicationerr.png
    The green arrow points to an Application Error.
    If you look in the text box below, you'll see "Faulting application name: MaxIm_DL.exe" and a bunch more text.
    Copy that text, and upload here.
    This may give us a clue if it is hanging/crashing on a device driver problem or something else.

    Gut feel on this is underlying hardware problem - like a bad cable, an unreliable USB hub (or one that doesnt like the ambient temperature), bad connection, operating system corruption, or an intermittent/pending failure of the filter wheel or camera electronics.
     
  3. James Pierce

    James Pierce Cyanogen Customer

    Joined:
    May 20, 2019
    Messages:
    62
    Sorry Colin, I should have been more specific when I said 'hang'. I think what is happening is that the camera itself is crashing, or hanging. That causes it to disconnect from MaximDL. That means that ACP just waits and waits and waits for MaximDL to finish whatever it was doing - probably the camera object is still left in some odd state from it's point of view? Bob Denny question there. To be clear the MaximDL application continues to run, and can be exited etc. The camera cannot be reconnected without a power cycle and now as of last night, a power cycle doesn't seem to be resolving the FW #6 error.

    If I disable the filter wheel it the config, it does still connect - but I don't know if it will stay stable in that state, and it's kind of not relevant apart from testing because the system is useless without FW control of course.

    I mentioned the 340m only because I wondered if there is a low level underlying driver or DLL that would give a clue. It could also just be a red-herring. Like I said above, the endless windows updates to seem to sometimes cause short term instability for various bits of the observatory, then they resolve with the next update just as silently. So let's ignore it unless it continues to occur with some regularity. Back to the problem...

    After reading through your answer I agree it feels like it might be hardware. I had the camera and filter wheel off the scope about ~5 months ago to give the filters and front CCD chamber class a blow off to get rid of some dust bunnies but I've not touched it since then so it's quite conceivable that there is some minor corrosion on a connector or similar. The camera itself is connected directly with a single high quality USB cable direct to the computer to try and avoid all the potential hub and interaction issues. We have just come through our summer here, so perhaps even just a bit of hot/cold expansion and contraction has caused something to shift a little.

    I'll see if I can get my onsite support to check each cable carefully and given them all a couple of plug / unplug cycles with a little contact cleaner to hopefully rule out any connection problem. I will also try and get them to get the serial numbers for all the bits in the camera stack from them at the same time.
    Frustratingly I was onsite at the dome last weekend. At worst I'll have to take a trip there personally soon.

    One though, you may have all the serials on file in your system. The camera setup was ordered via Peter Ward in June/July 2019 as an SBIG STX16803 Ultimate Package.

    Let me know anything else you can think of to check / try - If I have to make a trip I might as well get though as many things as possible.
     
  4. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    5,250
    Location:
    Earth
    Electro-static discharge can be a problem when cleaning filters etc. Best done on an anti-static mat. I've seen some "canned air" products generate high static electricity amounts, that can damage an IC or transistor, and this may not fail outright, but become flaky.

    Send me the pair of ACP logs so I can have a look.

    The SBIG Driver Check does have a DEBUG option, and you can turn on the debug logs for the filter wheel commands, and I could have a look at that, but it gets verbose fast.
     
  5. James Pierce

    James Pierce Cyanogen Customer

    Joined:
    May 20, 2019
    Messages:
    62
    Yup cleaning done carefully with consideration of grounding etc. The filter wheel itself has to come right out to give them a clean anyway!

    I'll try and dig up a couple of good examples from ACP.

    Let me know where I turn on the debug option and let's see if that gives you another clue.

    I was testing again in CCDOps just now. It connects to the camera fine and will take an image, but when you try and connect to the filter wheel it sort of appears to work (says it is moving) but then reports an unknown filter state down the bottom. If you try another filter move it just reports "CFW Busy" - In the setup filter wheel dialog there is a button on the right of the comm port setting which runs the CFW-L diagnostics... it does appear to be talking to the FW at some level because it reads back this?

    Firmware Version: 0X00FF
    Number Filters: 7
    Mem 00 = 0xFF (Status)
    Mem 01 = 0xFF (Filter)
    Mem 02 = 0xFF (CalMax)
    Mem 03 = 0xFF (CalMin)
    Mem 04 = 0xFF (Threshold)
    Mem 05 = 0xFF (Move Low)
    Mem 06 = 0xFF (Move High)
    Mem 07 = 0xFF (I2C Address)
    Mem 08 = 0xFF (I2CData)
    Mem 09 = 0xFF ()
    Mem 10 = 0xFF ()
    Mem 11 = 0xFF ()
    Mem 12 = 0xFF ()
    Mem 13 = 0xFF ()
    Mem 14 = 0xFF ()
    Mem 15 = 0xFF (Version)


    These are the times when I hate having a remote observatory, so much harder to just quickly check things like, is the filter actually attempting to move at all or not!
     
  6. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    5,250
    Location:
    Earth
    Ok, this may have to go to @Bill bill (At) sbig (dot) com for repair.
     
  7. James Pierce

    James Pierce Cyanogen Customer

    Joined:
    May 20, 2019
    Messages:
    62
    Looking at the ACP / Scheduler logs for the last couple of days the only error is the same as the one reported in MaximDL (Error 6)

    eg.

    08:32:36 Weather already connected
    08:32:36 OK
    08:32:36 Telescope already connected
    08:32:36 Waiting 5 seconds
    08:32:41 Connect ACP to the camera
    08:32:50 **Failed to connect to MaxIm and camera:
    08:32:51 Could Not Initialize Camera
    Error opening filter wheel
    Error (6) initializing filter wheel.
    08:32:51 **Script Error (Tracking has been stopped)**
    08:32:52 Source: ACP Observatory Control Software
    08:32:52 Message: ==== Startup failed, cannot continue
    08:32:52 Location: line 190 column 5.
    ACP console log closed 12-Mar-2021 08:32:53 UTC


    Going back in time a little to the nights where it appears to have failed mid session somehow here is a zip of a whole sequence of observations... The first ~1/3 all execute as expected. Then we see the "**Pointing update error from MaxIm DL 6:Exposure Aborted" and then finally

    14:50:11 **Script Error (Tracking has been stopped)**
    14:50:12 Source: Microsoft VBScript runtime error
    14:50:12 Message: The remote server machine does not exist or is unavailable: 'Camera.LinkEnabled'
    14:50:12 Location: line 956 column 5.

    Neither really gives much of a helpful clue though?

    You are very welcome to remote into my system sometime via team view and walkthrough / test anything you think might be helpful. Anyway as promised above, I will try and rule out simple a simple mechanical issue and then we'll take it from there.
     

    Attached Files:

  8. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    5,250
    Location:
    Earth
    Let's try to tee-up a TeamViewer session for Monday or Tuesday of next week.
    We're on Eastern time UTC-5 (although I think it changes to UTC-4 on Sunday)
    I'll need you to send me a Private Message (click under my photo) with your telephone number as backup, and Teamviewer info.
     
  9. James Pierce

    James Pierce Cyanogen Customer

    Joined:
    May 20, 2019
    Messages:
    62
    See your PM Colin ... Short version, bad news... replacement board hasn't solved the issue.
     

Share This Page