Staff Assigned: Tim AO-X not responding problem?

Discussion in 'Adaptive Optics (AO)' started by VictoriaSaucier, Oct 9, 2019.

  1. VictoriaSaucier

    VictoriaSaucier Cyanogen Customer

    Joined:
    Oct 1, 2019
    Messages:
    10
    I've been having difficulty getting my AO-X working reliably. I had written up a long post to share here and get people's ideas but while writing the problem down I may have figured it out, I think it was my mistake. I'll spend another night fighting with it before I post my issue.

    That said, I want to ask about a second AO-X issue I'm having:

    Sometimes the AO-X refuses to move at all. In MaxIm, AO calibration doesn't move the star, going to the setup tab and exercising does nothing. Power cycling the camera fixes this problem. The cables are tight, it's not a loose cable. When the AO-X isn't working, the camera and filter wheel still work, and the filter wheel is plugged into the AO-X which is plugged into the camera. I would think if it were a cable problem the filter wheel would also stop working. This problem occurs in both MaxIm and CCDOps. It does it whether I connect to the camera via ethernet or USB.

    Does this happen to others? Is this a sign of a larger problem?

    Setup, connected physically in this order:

    SBIG STX-16803 camera SN X14050351 (connected via Ethernet)
    FW5-STX
    SBIG AO-X SN 1405
    Planewave IRF90 Integrated Rotating Focuser
    Planewave CDK24
    AP3600GTOPE mount

    Software:
    Windows 7
    MaxIm DL 6.2
    ACP Expert 8.2

    I have all the latest camera drivers installed:
    sbigudrv.dll 4.99 build 7
    sbigu64.sys 2.41.0.1338
    sbigpcam.hex 2.46
    sbiglcam.hex 2.20
    sbigfcam.hex 2.25
    sbigfga.bin 2004.11.10

    stxusb.bf2 1.06
    stxga.bf2 2.54
    stx16803.bf2 1.00.05
    stx0340.bf2 1.00.04
     
  2. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    2,474
    I'm trying to figure out how you are guiding:
    a) Do you have an STX Guider attached to the Remote Guide Head port of the camera, so that you are guiding ahead of the filters?
    b) Are you using an STX-16803 camera with an internal guide chip (guider chip is after the filters in the main camera body)? (I think this is what you have - please confirm)

    Ok, let us know what you find.

    "Sometimes" - We'll need to figure out how the AO-X gets into this state.
    Let's try to focus on the circumstances where this happens.
    Is this when things are freshly powered on?
    Was MaxIm controlling it fine, then it stopped?
    Was this when ACP was running a scheduled job or during a "live observing" task?
    Does it work at the start of the night, and fail later on?
    Does it happen after the IRF90 rotates / refocuses?
    Does it happen after a slew / change in orientation of the OTA?
    Under what circumstances do you calibrate the AO? The guider? (Why are you doing it).

    A few troubleshooting suggestions:
    In MaxIm, View... Log Window. Right click in the log window. Turn on time stamps and debug messages.
    Run your tests. See if there is anything unusual in the log.
    You can upload your log file from (somewhere like):
    C:\Users\Victoria\Documents\MaxIm DL 6\Log

    If it is happening in ACP, please attach the relevant logs from ACP as well as the MaxIm log.

    Are you using the original power supply that came with the camera? (Some people run a different +12V/13.8V power supply to a number of devices.)
    If you are repeatedly power-cycling the camera, give it at least 10 seconds, preferrably 30 seconds for the power supply capacitors to fully discharge.
     
  3. VictoriaSaucier

    VictoriaSaucier Cyanogen Customer

    Joined:
    Oct 1, 2019
    Messages:
    10
    I hate the word "sometimes" too. I recognize I do not have enough information to diagnose the problem unless it's happening to others as well. I had posted to see if this is a "thing" that others are experiencing or if it's just me and I need to track it down. Clearly it's just me.

    I will do more homework.

    I need to mention that I stopped trying to get things to work for a couple years while I focused on a new business. The last I tried to use the AO-X was while there were known issues as confirmed by Alan and Douglas when SBIG and Diffraction were independent. I was having this non-responding problem years ago, and figured it was drivers/software or something.

    Only a week or two ago did I update everything and started to try to get the AO-X working.

    It typically goes something like this: Power things on, connect to camera in MaxIm, turn on coolers. Open observatory. Have dinner while the heat escapes the observatory and it gets dark. Start an ACP acquire image run to slew/rotate/rotate dome/call PWI to focus/update pointing. Abort ACP, make sure tracking stays on. Then forget about ACP while I play in MaxIm. Do a locate exposure, see a nice star. Adjust duration/binning if necessary. Calibrate AO. See star not move at all. Test AO in both X and Y, hear crickets from the AO. Disconnect in MaxIm. Unplug camera. Wait for 15 seconds or so, plug it back in. Wait for camera to start up. Reconnect in MaxIm. Test again, hear noise, rejoice. Calibrate in MaxIm, see star move, but even if it's a bright (like mag 2) star with a dark background, watch as MaxIm fails to identify the star in all the exposures. Do it two or three times until MaxIm calibrates.

    I have been doing a calibrate each time because I have taken exactly one exposure where the AO appeared to work, and that was last week. (And then I noticed part of the camera frosted over, and I realized I needed to bake the desiccant plug; I'm not frosting now) I have been trying to image from within MaxIm directly, and more often than not the star ends up going off out of the field the AO is able to correct. Since I have guided without the AO in the past, my polar alignment is good and I have encoders to help with PE, I have assumed the problem was with the AO and I did not go back to make sure the mount is guiding correctly otherwise. *Right now I'm operating under the assumption that the AO is working and that I need to make sure the mount is guiding correctly.*

    But back to the non-responding problem: After I've power cycled things and confirmed the AO is moving, I have slewed and rotated and refocused and controlled things from MaxIm and ACP, had the dew heaters and fans come on and go off, and have NOT seen any of those things make the AO stop working. I would almost dare say it's a problem from a freshly-powered up state. Once it's powered up it's either going to work or not. I have not repeatedly plugged it in, checked it, unplugged it, waited, repeat, in order to diagnose that statement.

    I will log things and do more work to diagnose.

    To answer a couple of your other questions: I'm using the internal guide camera. I have a 25-amp pyramid power supply I use for the camera/focuser/rotator/telescope fans/dew heaters. I have a separate supply for the mount. The connections all the way to the camera are solid and reliable and of a larger gauge than necessary. It's supplying 13.3v to the camera (measured by multi-meter at the camera and at the power supply while under load) I have not noticed a power-related problem with anything, and do not believe I'm coming anywhere close to 25 amps. I have another large power supply I could use, if you suspect power.

    Thanks for your help.
     
  4. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    2,474
    Thanks for the thorough explanation Victoria. I completely understand your frustration.

    A few thoughts (you've probably already thought of these), but I figure for completeness, I'll share:

    Put the camera stack (STX, FW, AO) on its own independent power, preferrably the original power supply. Dew Heaters and their controllers sometimes generate RFI and sudden loads to the supply; inductive loads like stepper/servos in the IRF90 can produce back emf and spikes. All of which will potentially be experienced by the Camera stack. Going back to the original SBIG power supply eliminates this possibility. If it has no impact after completing further testing, then fine, go back to everything on the Pyramid. Am glad to hear 13.3V is getting to the camera. The STXs from that time period need the extra juice.

    AO exercising - CCDOPS or MaxIm should be able to reliably exercise the AO 100% of the time, when the camera is freshly powered up and CCDOPS is launched into the AO... Exercise, with 0.2s (200ms) movements. You might want to use the laser-pointer trick mentioned in the manual to tell if the AO is moving properly.

    @Tim may have some suggestions on checking that the optical paddle is moving freely and properly. (This is a workbench task). It is possible that the gimbal is sticking a bit, either due to contamination, misalignment of parts, or thermal effects.

    I recommend that you make sure MaxIm has only one boss per session. Either use it manually (you're the boss), or under ACP control (and don't touch it). The MaxIm script interface used by ACP and the GUI you use could be competing/putting something into a funky state. Resuming ACP after manual activity may not inform ACP of any state changes that have happened. It may be safer to launch fresh, eg let ACP run the startup script again.

    Calibration in MaxIm - Put your rotator at PA=0, I'll assume the camera rows are aligned with X axis (RA) of the Astro-Physics mount.
    In MaxIm, calibrate the AO. After this is done, you should never need to Calibrate the AO again, unless you have disassembled the hardware (eg removed AO from FW etc).
    Next, calibrate the guider.
    Assuming MaxIm is aware of the rotator position, it should be calculating the correct bump/guiding pulses go-forward.
    From this point forward, guiding with AO should be repeatable in MaxIm; bumps happening only when the AO runs out of travel, and in the correct direction..
    You'll want to try a few sessions of manually-started guiding from a cold start eg shut all down, then power up, launch MaxIm, connect/cool, slew somewhere, start guiding process.
    Once all is good w MaxIm, then move on to ACP.

    It's been a while since I've done the ACP version of the rotator/guider/AO setup. I recall there is a section in the ACP help about the process. I'd leave ACP out of this until we determine if there is a hardware issue or some misconfiguration or something I'm not thinking of.

    ACP's startup script can be configured to automatically do things like fire up the coolers etc. It might be worth talking to @Bob Denny for a bit of assistance with that.

    Is your setup internet accessible - eg if we needed to, could we arrange a TeamViewer session with you?
     
  5. Tim

    Tim Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    1,626
    It's possible if it never worked correctly. Take the AO-X lid off and gently move the gimbal in all 4 directions and make sure it floats freely with no scraping or binding. It should move smoothly.

    I did not see anywhere in your comments where you tested the AO-X with only the camera. Have you tried this? Do you have a power supply plugged into the FW5-STX as well to provide additional auxiliary power?
     
  6. VictoriaSaucier

    VictoriaSaucier Cyanogen Customer

    Joined:
    Oct 1, 2019
    Messages:
    10
    Wow! Thank you for much more advice than I expected or deserve! I'll keep it all in mind next time it's not overcast.

    Yes, the machine is always connected and is running TeamViewer.

    Thanks again!
     
  7. VictoriaSaucier

    VictoriaSaucier Cyanogen Customer

    Joined:
    Oct 1, 2019
    Messages:
    10
    @Tim: I have not tried the AO-X without the filter. I will add it to the list of things I'll try next session! I am not powering the filter wheel. If that is something I should be doing I will do that! Thank you!
     
  8. VictoriaSaucier

    VictoriaSaucier Cyanogen Customer

    Joined:
    Oct 1, 2019
    Messages:
    10
    An update, of sorts:

    On the non-responding issue, I have not tried to repeat the problem, I have nothing new to report other than I have not power-cycled anything in three days and the AO is still responding after quite a bit of usage. It feels like it's locking up only on power-up, and only sometimes, but again I don't have direct evidence to support that, only the negative it-hasn't-locked-up-and-I've-been-using-everything.

    On the other hand: I have the AO working otherwise from within MaxIm, at different PAs, on both sides of the mount! And this is on a mostly-cloudy, moonlit night! (I noticed that a remarkable amount of light from Vega can come through some clouds) Thank you!

    I do not have the AO working reliably in ACP, but after more time trying to get it working there if I fail I'll go over to the ACP forums and bug Bob. (ACP-controlled guiding with the AO is working just fine up until the mount itself needs to be jogged a little, leading me to believe something is set wrong, like the guide sensor angle (although I think I have that measured correctly). I'll work on it)

    One more thing before I go and leave you alone... When I calibrate with the AO-control/Locate/Calibrate Drive, presumably the guide sensor's angle is measured relative to the mount. Is that stored/does that show up somewhere? Are the guider settings on the camera control/guide/settings used at all for AO guiding? I have to assume at least some of them are, but maybe not. I don't want to have something set in there that's messing up ACP.

    Thank you guys again for your help.
     
    Last edited: Oct 12, 2019
  9. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    2,474
    In the MaxIm Settings directory are files that retain camera settings and related items.

    If you wish to experiment, use the File.. Configurations option to save a "known good" version of your settings, or a "pre-experimentation" version.
    Then play to your heart's content.
    Save the final Configurations as "My new happy ones" when you are done.
    If you mess up, you can reload the "known good" ones at any time, and it will restore everything.

    On my home observatory, I have a Configuration called "ACPgood20181210" for when I last confirmed I had all working with MaxIm and ACP.
    Then when I experiment to test things, I change the settings, and when done, revert to the "ACPgood" ones to restore normal operation.
    ACP has a similar facility.

    Another suggestion: with the rotator homed to PA=0,grab a guide camera image and a main camera image. Use Analysis... Pinpoint to platesolve both images. Then you will get the true scale and orientation info. Write it down in case you need to figure it out later.

    The AO calibration retains how much the tip/tilt moves the optical puck. In other words, a certain amount of current to the deflector coils makes the starlight deflect maybe 40 pixels across the guide sensor's field of view in each of x and y.
    The guider calibration retains how much the mount moves in each direction based on guide pulses. e.g. a guide relay pulse of 10 seconds makes the star move 400 pixels.
    The last part is that the software knows that if the AO coils can only move 40 pixels, when we hit that limit of the AO, we have to send a guide pulse of say 1 second to nudge the scope by 40pixels (1/10 x 400 = 40). This is the "bump".
    Am oversimplifying a bit.
     
  10. VictoriaSaucier

    VictoriaSaucier Cyanogen Customer

    Joined:
    Oct 1, 2019
    Messages:
    10
    Thanks again Colin.

    Last night I was going to play with settings but I ran into the AO not responding issue and after power-cycling three or four times to no avail I ended up trying to troubleshoot that instead.

    I used a different power supply, another Pyramid 35a power supply set to 13.5v.
    I disconnected the filter wheel from the AO so that I could rule out the filter wheel causing a problem.
    The AO was plugged into the camera with the original I2C cable.
    The camera had nothing else plugged into it.
    When I plugged in the camera, SOMETIMES (I know) the LED on the AO would blink fast. Sometimes it would stay on. Sometimes it would blink on and off at 4.5s on, 4.5s off, 4.5 on, etc.
    I made sure to keep the camera unplugged for 30s between tests.
    I then tried plugging in an ethernet connection to the camera, and repeated the tests, with the same results. Each time, I connected to the camera from MaxIm and tried to exercise the AO. As expected, the AO only responded when the LED was blinking quickly. The camera itself appeared to work fine regardless of the state of the AO.

    Is this a send-it-in-for-repairs thing?

    Thank you!

    Edit: I can probably find the original power supply to try that, if you think it will matter. Unfortunately, however, I cut/modified the original cable going to the camera to put powerpole connectors on it to attach it to my power distribution at the telescope. I will gladly unmodify the cable and put it back on the original supply if you guys believe it's a power issue. Since the camera (and filterwheel, when attached) work very well, I don't believe it's a power issue. But you guys know best.
     
  11. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    2,474
    It sounds like the AO-X is not resetting properly.
    @Tim may have some idea - to me it sounds like a hardware issue.
     
  12. Tim

    Tim Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    1,626
    I don't have any further suggestions to add at this point. I would recommend you contact @Bill Lynch and arrange an RMA for him to have a look at the AO.

    ---------------------
    Bill Lynch – SBIG Service & Repair Center
    3769 Constellation Road, Unit D
    Lompoc, CA 93436


    Phone: (805) 308-6979

    Note: Please contact our authorized service & repair center for an RMA number prior to returning your equipment.

    ---------------------
     

Share This Page