Changing background values on STi images. Causing guiding problems and star fades.

Discussion in 'STF Series CCD Cameras' started by JoshuaHufford, Apr 22, 2017.

  1. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,077
    Location:
    Earth
    GuiderTracking120.fit in the Full Size Guide Images No Calibration Bin 1x1 clearly shows an issue with downloading the STi image.
    I'll take a look at some of the others and see if I have any insights.
     
  2. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,045
    Location:
    Ottawa
    Sorry Joshua, I've been completely swamped, and forgot to update you when I got sidetracked. I am looking through the logs, and trying to piece together what's going on. I'll get back to you tomorrow morning with a more detailed update.
     
  3. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,077
    Location:
    Earth
    Actually, all of them in that file are really bad.
    Joshua, have you got a Powered USB 2.0 hub you could try putting between the PC and STi ? Am wondering if there is a voltage drop on the USB bus or something like that.
     
  4. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,077
    Location:
    Earth
    Only 119 looks normal, yet there are no stars.
     
  5. JoshuaHufford

    JoshuaHufford Cyanogen Customer

    Joined:
    Oct 10, 2014
    Messages:
    510
    Hi Adam, thanks for getting back to me. I understand you are busy and this probably is low on the priority list, it is just nice to know it is at least getting some attention.

    Colin, the images I took this night were the worst by far, the ambient temp was a bit higher than normal but other than that no change in my setup. For this test I did not have a hub, just a single cable to the STi, but I have tried a hub in the past and still would get images with increased noise. Next clear night I'll probably try to put the hub back in line so the camera has full power right at the pier.
     
  6. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    8,077
    Location:
    Earth
    Josh - the thing I find weird is that the problem comes on as you go across the rows, and continues in bands down the chip. Very weird.
    Any chance the USB cable runs near a power cable or motor control wiring? Is the camera near any RF source?
    Do you have ferrites on the cable?
     
  7. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,045
    Location:
    Ottawa
    Alright. So I've dug into your logs, and it looks like you were taking continuous 2-second exposures with your camera. Most images had their exposures ended within 50 ms of the requested exposure duration, but I've noticed a few frames where there are large gaps in our status polling command, immediately preceding the call to end the exposure — sometimes in excess of a second or so. That's really the only indication that something isn't quite right (from the log files). Worse yet, I can't directly correlate images to log-calls, because it looks like logging hit the cyclical limit while collecting data — so I have no way of knowing whether those frames are the ones exhibiting higher than average pixel values. I took the liberty of plugging in an ST-i here, to see if I could reproduce the log results with bad frames — and while I did see some frames with moderately higher background ADU counts, I didn't see anything in the logs hinting a delay in status polling to the camera, or even a background jump as large as the ones you saw.

    I've spoken with our hardware team, and the ST-i (being an interline sensor) is designed to immediately transfer the image to the frame buffer as soon as the camera's internal exposure counter hits zero, and is read out while the shutter is closing (the benefits of interline sensors) without delay. My operating theory was that the camera was getting into a state where the shutter was closed, but the frame hadn't transferred to the frame buffer, and it was sitting there accumulating dark current — however, nothing in the design of the camera seems to support that hypothesis on inspection.

    Long story short: the logs and firmware weren't as illuminating as we'd hoped. While this could be a digital problem, I'm not yet convinced the error doesn't have a root in exposure control logic, or possibly that it's just a power supply issue. ST-i cameras draw very close to full power from USB, and it's possible that (with a third-party cable that uses a low-gauge power wire) there could be enough of a voltage drop that the camera isn't drawing full power. It could also be that some notebooks don't have USB ports that meet USB 2.0 spec, and can't offer full power either. Doug suggested (if you haven't already) plugging the ST-i directly into the controlling computer via the USB cable we provided (which was chosen specifically to meet our power draw needs), and failing that, plug it in with a powered hub (to guarantee the hub can meet the power draw).

    If neither of those solutions work, it's possible there's something wrong with the board, but Bill is on vacation this week, so we'd have to chat with him when he gets back.
     
  8. JoshuaHufford

    JoshuaHufford Cyanogen Customer

    Joined:
    Oct 10, 2014
    Messages:
    510
    Hi Colin. No RF anywhere that I can think of, and I ran my SSAG camera through the exact same routing for several years, also my ST8300m uses the same routing with no problems with RF noise, although I know that isn't a conclusive test. I don't have any ferrite beads on the cable but I would be willing to try.

    Adam, thanks for the reply. I'm not sure if you have had a chance to read this entire thread from the start, I know you came in on this later on but I'll try to re-cap what I've done and tested so far. The USB cable I normally use is the proper gauge and is the 15ft, although I do have a 3ft extension, that is the only way I can reach the camera and still have the PC inside the control room. It is plugged directly into the motherboard of my PC, it is not a laptop or notebook PC. I know this is beyond spec, however I have also tested with the single 15ft cable that came with the camera connected directly to the motherboard of the PC, and I have also tested with a powered hub (StarTech industrial) at the pier with a short cable between it and the STi with the same results, no change. I added the powered hub shortly after I got the STi because I was beyond the specs. I did the test images without the hub because I wanted to eliminate as much hardware as possible from the equation, and I never saw a change in results regardless of how the camera was connected to the PC.

    You mentioned that " logging hit the cyclical limit while collecting data". Is there a way the limit can be increased? Or if I stop and make new log files more often will it prevent this?

    You also mention " My operating theory was that the camera was getting into a state where the shutter was closed, but the frame hadn't transferred to the frame buffer, and it was sitting there accumulating dark current". It was my understanding that during normal exposures the shutter stays open the whole time, it does not close after each exposure. So I'm a bit confused by this.


    I have a few options for further testing which I'll be happy to do, let me know if any or all of these would be helpful. My friend who I share an observatory with also has an STi and it has the exact same problem with the occasional frames with high background noise. I could use his camera on my PC, or I could install my copy of Maxim on his PC and test his camera with logging turned on. We also do have a third PC in the observatory, a small ASUS box PC that we used for controlling shared observatory hardware, I could test either camera from that PC. Just let me know.

    Thanks.
     
  9. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,045
    Location:
    Ottawa
    If you're certain the ST-i is getting full power draw, then we're quickly running out of rational explanations for irrational camera behavior.

    Re: the logging file size limit: it's hard coded. I advise doing short bursts of imaging and clearing out the log files between those bursts (keeping copies of course). If you can get a bad frame within 5 or so images with a fresh log file, I can see if the two phenomena are linked.

    Re: my operating theory: it does normally stay open during guiding operations; but even with that, it can sit in the frame buffer, undigitized, accumulating dark current, and amp glow. We've addressed issues with some cameras that have had exposure control logic producing weird image artifacts in the past, and until yesterday I didn't want to rule out that possibility. Basically, I was hoping this was a firmware bug, and that doesn't appear to be the case — at least, not obviously so.

    Let's start by trying to get the log-to-frame correlation sorted out. If there's some correlation between those two phenomena, it will broaden our search a bit. Failing that, try the other PC for good measure. It wouldn't be the first time a bad USB port has been to blame for odd camera behavior...
     
  10. JoshuaHufford

    JoshuaHufford Cyanogen Customer

    Joined:
    Oct 10, 2014
    Messages:
    510
    OK I will try to get a log for you that has a bad image in the first few frames. It would be nice if the code were changed so that wouldn't be a problem.

    Unless you would rather I not I'm going to plug the STi back into the powered hub at the pier.
     
  11. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,045
    Location:
    Ottawa
    I am developing a logging system that can create multiple log files, but they get so big, so quickly, that it can be dangerous to forget you have it on — it's also taken a back-seat to some more pressing development/tech support matters. Go ahead and plug the ST-i back into the pier, and I'll keep an eye out for those new logs.
     
  12. JoshuaHufford

    JoshuaHufford Cyanogen Customer

    Joined:
    Oct 10, 2014
    Messages:
    510
    Thanks, weather has not been the best and I've been super busy lately but hopefully I'll be able to test again soon.
     
  13. JoshuaHufford

    JoshuaHufford Cyanogen Customer

    Joined:
    Oct 10, 2014
    Messages:
    510
    Hi Adam, I have some new images and logs for you, hopefully these will be of more use to you. I only exposed a few images at a time and then renamed the current log file so a new one was created each time, I added a letter beginning with a and worked my way up for each new log file. I didn't do this with the guide images since they each have their own file name, hopefully you can match them up with the logs. I did see some bad images within the first few frames of each series that I did.

    2 things have changed since my last test, I upgraded the version to 6.14 of MaximDL, and I plugged the STi back into the powered hub at the pier, so it is now running from an industrial StarTech powered hub with a 3ft quality USB cable, if that doesn't supply enough power then I don't know what will. Let me know if you need anything else.

    Thanks.
     

    Attached Files:

  14. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,045
    Location:
    Ottawa
    Thanks Joshua, I'm looking at them now.
     
  15. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,045
    Location:
    Ottawa
    Am I correct in assuming that GuiderTracking042.fit is the first image in "Logs 10 14.txt"?
     
  16. JoshuaHufford

    JoshuaHufford Cyanogen Customer

    Joined:
    Oct 10, 2014
    Messages:
    510
    It should be yes.
     
  17. JoshuaHufford

    JoshuaHufford Cyanogen Customer

    Joined:
    Oct 10, 2014
    Messages:
    510
    BTW I did also have my ST-8300M connected and cooling while doing this test. I didn't think to leave it disconnected while gathering these images, I hope that doesn't mess you up.
     
  18. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,045
    Location:
    Ottawa
    It *should* be OK. I'll be counting CC_START_EXPOSURE2 calls, so as long as you weren't exposing images with the 8300, we should be in business.
     
  19. Adam Robichaud

    Adam Robichaud Lead Developer Staff Member

    Joined:
    Sep 29, 2014
    Messages:
    1,045
    Location:
    Ottawa
    Unfortunately, I don't see any correlation between logged exposure durations and bad images. There were three images (GuiderTracking053.fits, GuiderTracking054.fits, and GuiderTracking055.fits) that would have shown up back-to-back in the logs if that was the case, but I never saw so much as a pair of sequential exposures hinting that — and none of those frames lined up with problematic logged exposure durations in the log files. I'm going to call the odd durations a red herring for now, and talk things over with our hardware team. I'll get back to you shortly with more thoughts.
     
  20. JoshuaHufford

    JoshuaHufford Cyanogen Customer

    Joined:
    Oct 10, 2014
    Messages:
    510
    Well I was hoping for better new, thanks for continuing to look into this. If there is anything else you would like me to do on my end please let me know.
     

Share This Page