Camera Disconnects Randomly

Discussion in 'STC Series CMOS' started by Oscar Echeverri, Nov 9, 2020.

  1. Oscar Echeverri

    Oscar Echeverri Cyanogen Customer

    Joined:
    Sep 2, 2020
    Messages:
    35
    Location:
    White Lake, Ontario
    Hello,

    I've been using a brand new SBIG STC-7 I just received with MaximDL Pro 6.24. I'm getting somewhat frequent camera disconnects. There seem to be 2 scenarios that the disconnects occur, 1 which is more easily reproduced than the other:

    1) Any time a frame is taken with a subframe. For example, if I turn subframe on under capture, select a region around a bright star and expose, most times the camera will disconnect. If it doesn't, the image returned is blank (I'm not sure if it's actually blank, or just imaging a random region that doesn't happen to have a star in it). This issue also frequently occurs when I try to use autofocus -- when the algorithm tries to take exposures to measure HFD, the camera disconnects. I disabled subframes for the autofocus exposure, and that resolved this particular issue. So this manifestation at least seems related to capturing subframes. Dialog pop ups saying something like "Unexpected disconnection (-4)" or something similar, I don't recall the wording.

    2) Random disconnections -- this one is a little trickier, because I've not been able to pinpoint what makes this one happen. But as an example, I programmed a sequence that LRGBHa exposures what were supposed to take around ~4 hours. I triggered the sequence and left my observatory to do its thing. When I checked in 4 hours later, I had that -4 error dialog and it appeared as if though the sequence had only run a for about 2 hours before MaxIm lost connection with the camera.

    I've experienced both of these scenarios with on 2 different computers -- sometimes I can click the Connect button again under camera control setup and it'll reconnect -- other times if I click Connect it'll try, and then show the -4 error dialog again, and a third scenario that I see from time to time is I get an error saying something like "camera with S/N: xxxxxxx could not be found". When I hit this last scenario, I can only reconnect the camera by power cycling it and then re-trying the connection.

    Any help would be greatly appreciated :) I like to let my rig run primarily unattended, so my tolerance for these random disconnections is pretty low.
     
  2. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    7,379
    Location:
    Earth
    Hi Oscar,
    Here's some common things to check first:

    In MaxIm, check that you have the Camera 1 and Camera 2 (if used) [OPTIONS] button settings for enabling threading turned on:
    threading.png

    Windows Power Management sometimes can cause USB communication failures.
    There are a couple of things to check.

    Windows Power Management - USB Selective Suspend - this should be OFF (disabled) to prevent Windows from powering down USB hardware.
    usb_suspend.png

    For all devices, COM ports (USB/RS232 adapters), USB hubs, USB root hubs, PCIe devices, turn off [] Allow the computer to turn off the device to save power:
    USB_power.png

    You might want to save your Windows Power Plan with a new name after making these changes, sometimes that helps.
    We've learned that Windows Updates, Windows Feature Updates etc sometimes REVERT these settings, and that can cause trouble to show up later after a Windows update.

    Let us know how it goes.

    We'd like to take a look at the error messages that occurred, in case there is anything else going on.
    Please use the [Upload a File] button to the lower right and attach your recent MaxIm logs.
    The logs have names like "20201109.log" and are usually found somewhere like:
    C:\Users\your_name_here\Documents\MaxIm DL 6\Settings\Log
     
  3. Oscar Echeverri

    Oscar Echeverri Cyanogen Customer

    Joined:
    Sep 2, 2020
    Messages:
    35
    Location:
    White Lake, Ontario
    Hi Colin,

    Thanks for the quick reply! I had threading already enabled in MaxIm as recommended -- I've made the recommended changes to power management settings in Windows 10.

    I've attached the logs for my last 3 sessions on my main observatory PC.
     

    Attached Files:

  4. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    7,379
    Location:
    Earth
    Thanks Oscar.
    I forgot to mention that you should power-cycle everything after making the Windows Power Management changes.
    e.g. turn it all off and on again.
     
  5. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    7,379
    Location:
    Earth
    Oscar - we've had a quick look at the subframe problem, and we are going to investigate further. This may take a few days.
     
  6. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    9,932
    I've replicated the error.

    This used to work! It looks like something got broken the last time we updated the driver. Adam is currently working on it, and we should have a solution shortly.
     
  7. Oscar Echeverri

    Oscar Echeverri Cyanogen Customer

    Joined:
    Sep 2, 2020
    Messages:
    35
    Location:
    White Lake, Ontario
    Perfect -- thanks so much!
     
  8. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,012
    Location:
    Ottawa
    We've got a working fix that we're preparing for release (we have to validate it first). If you'd like to try the beta to assist in those efforts, I'd be happy to set you up with it.
     
  9. Oscar Echeverri

    Oscar Echeverri Cyanogen Customer

    Joined:
    Sep 2, 2020
    Messages:
    35
    Location:
    White Lake, Ontario
    Hi Adam!
    Sure I wouldn't mind giving it a try
     
  10. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,012
    Location:
    Ottawa
    I sent you a PM with the details. Let me know if you experience any issues.
     
  11. Oscar Echeverri

    Oscar Echeverri Cyanogen Customer

    Joined:
    Sep 2, 2020
    Messages:
    35
    Location:
    White Lake, Ontario
    Hello,

    Quick update ... so the beta driver seems to have resolved the subframing problem..

    I followed Colin's suggestions for USB/power management tweaks -- but I did a single 5 minute test exposure the night before last, and seems to have failed when downloading the image from the camera:

    04:29:13*4 Filter wheel moving to position 3
    04:29:13$5 Guider algorithm 'Single Star' initialized
    04:29:13$5 Acquiring guide target
    04:29:13$5 Using target position (494,364)
    04:29:16$5 Guider Tracking Started
    04:29:18*4 Started 300.000s 3216x2208 exposure, binned 1x1, ROI (0, 0)
    04:34:19*4 Completed 300.000s 3216x2208 exposure, binned 1x1, ROI (0, 0). <=== looks like exposure finished
    04:34:22$14 DLImaging: Retry 1/3 :: Encountered an API error (255) <==== failed to download image?? MaxIM didn't show the image
    Unknown Error
    04:34:22$14 DLImaging: Retry 2/3 :: USB Comm Error: 32
    04:34:30$14 DLImaging: Retry 3/3 :: Promise timed out.
    04:34:30$14 DLImaging: Retry 4/3 :: Promise timed out.
    04:34:30$3 Disconnected from filter wheel 1 "DL Imaging+FW"
    04:34:30$3 Disconnected from camera 2 "ASCOM"
    04:34:30$3 Disconnected from camera 1 "DL Imaging"
    04:39:16*3 Connecting cameras - thread ID 20304
    04:39:18#3 Failed to connect to camera 1 "DL Imaging" (CCDPlugDLImaging.dll), error -727226815. <==== tried to reconnect to camera, but failed
    04:39:18$14 DLImaging: OpenCamera(): Querying for cameras
    04:39:18$14 DLImaging: Attempting to Connect to camera with S/N 692518914 ... [Failed] :: Serial Number not found
    04:39:18$14 DLImaging: Could not find your camera w/ S/N: 0. <==== error I talked about in initial post
    Please verify your camera can be discovered in the Advanced Setup dialog before proceeding.
    04:39:42*3 Connecting cameras - thread ID 20304
    04:39:43$3 Connected to camera 1 "DL Imaging" (CCDPlugDLImaging.dll v6.24.2.0)
    04:39:43$14 DLImaging: OpenCamera(): Querying for cameras. <=== power cycled camera and was able to reconnect
    04:39:43$14 DLImaging: Attempting to Connect to camera with S/N 692518914 ... [Success::USB]
    04:39:43$14 DLImaging: Attempting to Connect to camera with S/N 692518914 ... [Success::USB]
    04:39:43$14 DLImaging: Retry 2/3 :: Encountered an API error (7). <==== ???
    Client Error :: Invalid Parameter
    04:39:49$14 DLImaging: Aluma Camera Flag State: 0
    04:39:49$3 Connected to camera 2 "ASCOM" (CCDPlugASCOM.dll v6.21.3.0)
    04:39:50$3 Connected to filter wheel 1 "DL Imaging+FW" (CCDPlugDLImaging.dll v6.24.2.0)
    04:40:06$3 Disconnected from filter wheel 1 "DL Imaging+FW"
    04:40:06$3 Disconnected from camera 2 "ASCOM"
    04:40:06$3 Disconnected from camera 1 "DL Imaging"
    04:40:11$9 Disconnected from telescope
    04:40:11*1 MaxIm DL Version 6.24 exited <==== shut down for the night cuz I had to work the next morning :)


    I've annotated the snippet in the log that I think is relevant.. attached the full log, but you'll notice it's mired with me trying to figure out auto guiding in MaxIm, which... I think I eventually did, and tried the above exposure to test... which failed :(

    Any help appreciated.
     

    Attached Files:

  12. Oscar Echeverri

    Oscar Echeverri Cyanogen Customer

    Joined:
    Sep 2, 2020
    Messages:
    35
    Location:
    White Lake, Ontario
    Hello -- just wondering if there's any word or thoughts on the random disconnections? Like I said -- the sub framing issue seems resolved with the beta drivers provided, but still seeing the odd random disconnection.

    See additional attached logs...

    Here's the snippet where it happened:

    20:00:46*4 Started 5.000s 1608x1104 exposure, binned 2x2, ROI (0, 0)
    20:00:51*4 Completed 5.000s 1608x1104 exposure, binned 2x2, ROI (0, 0)
    20:01:09*4 Started 5.000s 3216x2208 exposure, binned 1x1, ROI (0, 0)
    20:01:14*4 Completed 5.000s 3216x2208 exposure, binned 1x1, ROI (0, 0)
    20:01:14$14 DLImaging: Retry 1/3 :: Encountered an API error (255)
    Unknown Error
    20:01:17$14 DLImaging: Retry 2/3 :: USB Comm Error: 32. <=== Error
    20:01:25$14 DLImaging: Retry 3/3 :: Promise timed out.
    20:01:25$14 DLImaging: Retry 4/3 :: Promise timed out.
    20:01:25$3 Disconnected from filter wheel 1 "DL Imaging+FW"
    20:01:25$3 Disconnected from camera 2 "ASCOM"
    20:04:45*1 MaxIm DL Version 6.24 started on OBSERVATORY
    20:04:49>6 View, Camera Control Window
    20:05:04*3 Connecting cameras - thread ID 9932 <=== Tried to reconnect to cameras
    20:05:06#3 Failed to connect to camera 1 "DL Imaging" (CCDPlugDLImaging.dll), error 1129216619
    20:05:06$14 DLImaging: OpenCamera(): Querying for cameras
    20:05:06$14 DLImaging: Attempting to Connect to camera with S/N 692518914 ... [Failed] :: Serial Number not found <==== failed to reconnect
    20:05:06$14 DLImaging: Could not find your camera w/ S/N: 0
    Please verify your camera can be discovered in the Advanced Setup dialog before proceeding.
    20:05:51*3 Connecting cameras - thread ID 9932. <=== power cycled camera and retried.
    20:05:53$3 Connected to camera 1 "DL Imaging" (CCDPlugDLImaging.dll v6.24.2.0)
    20:05:53$3 Connected to filter wheel 1 "DL Imaging+FW" (CCDPlugDLImaging.dll v6.24.2.0)
    20:05:53$14 DLImaging: OpenCamera(): Querying for cameras
    20:05:53$14 DLImaging: Attempting to Connect to camera with S/N 692518914 ... [Success::USB]
    20:05:53$14 DLImaging: Attempting to Connect to camera with S/N 692518914 ... [Success::USB]
    20:05:53$14 DLImaging: Retry 2/3 :: Encountered an API error (7). <=== I see this a lot in the logs???
    Client Error :: Invalid Parameter
    20:05:53$14 DLImaging: Aluma Camera Flag State: 0
    20:06:21*4 Filter wheel moving to position 3​

    Always seems to happen when downloading an image from the camera.. power cycling the camera fixes the issue, but that sucks for me when my observatory is ~100m from my control room :) or if it happens while I'm not watching it.
    This occurred now with a different USB-Hub as I wanted to rule that out as a potential problem..

    I was able to watch it happen, and when the download bar reaches completion, MaxIm seems to hang for a moment (Windows flags it as Not Responding), and then some time later 10-15 seconds the image that opens in MaxIm is tiny and blank, and then the camera disconnects...
     

    Attached Files:

  13. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,012
    Location:
    Ottawa
    So the important thing I'm seeing in the logs is that we get an unknown error from DLAPI before the connection craps out. Let's look at this from 2 angles:
    1. What version of the STC firmware are you running?
    2. Let's collect a driver log to diagnose what call is causing the problem.

    Can you turn on DLAPI logging via Camera Control > Setup tab > Camera 1 > Setup Camera > Debug Logging, and hammer the camera with a combination of dark and light frames during your off hours to try and capture the error? We're aware of the Invalid Parameter error, but we haven't seen it cause any issues that weren't safely handled in the plugin, so it hasn't been a priority investigation (yet). The fact that you have to power cycle the camera hints there's something going on in the firmware, but we like knowing the exact call before bringing it to hardware for investigation.
     
  14. Oscar Echeverri

    Oscar Echeverri Cyanogen Customer

    Joined:
    Sep 2, 2020
    Messages:
    35
    Location:
    White Lake, Ontario
    Hey Adam,

    Looks like STC FW rev3 --- attached screenshot from DLConfig.

    I'll start an exposure sequence now with debug logging turned on -- I'll let it run couple hours and post logs.
     

    Attached Files:

  15. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,012
    Location:
    Ottawa

Share This Page