AC2020bsi boots up producing invalid images with weird histograms

On 2 occasions now, my AC2020bsi takes very weird images on powerup. Several are attached (“crapX.fit”), as is a photo of my Maxim DL screen, showing the histogram of one of them ("crap4histo.jpg")

.

This has happened twice, on July 29/30, and Sept 01/02, with 11 nights of proper function in between. I didn’t do anything different on these two nights, and I don’t know how to trigger the problem on demand. On both nights, I first noticed the problem when I started taking images on the sky. I tried various things like taking dark instead of light images, switching from high gain to low gain, 1x vs 2x binning, exiting and restarting MaxIm, etc., to no avail. The fix comes when I disconnect the camera from MaxIm, cycle the power on the camera, and reconnect, etc. (image “good1.fit”). It is not necessary to restart MaxIm or reboot the computer.

 

Colin:

Attached is the camera info (and guider, for completeness).

The camera, and nothing else, is powered by the original power supply.

I believe the shutter opened. Here's why: on image good1, there is a star at ~342, 45. On image crap 4, I can barely make it out, but there is a hint of a star at that position (just a few pixels, but much higher ADUs than the surrounding). I noticed it on crap4 without deliberately looking for it (for whatever that is worth!). It shows up best using "screen stretch" "low". Now looking for it, I can see it in image crap3 as well.

I operate from 100' away: is my memory correct that I would hear a click when the shutter opens? (Is there any other way?) If so, if/when this recurs, I could shoot "continuous" and go listen for clicks. But I find the info in the above paragraph pretty convincing.

HTH, g.

 

By happenstance, on both nights I noted some temperature info. I ran the camera at -15 C, and it was reporting 64% cooler power and heatsink temp of 22.4 (same numbers on both nights, which is remarkable). That was at the start of my "science run", i.e. shortly after these startup problems were fixed by cycling the power.

So ambient was moderate compared to hottest nights, when heatsink temp would be nearly 30 and/or I could not run as cold as -15.

I will continue to log temperature info.

g.

 

Since my previous reports, I updated MaxIm to 6.40 and DLimaging driver to 2.7.1.0 before any of the instances reported here.

I have had two more instances of my AC2020bsi taking weird images on power up. I can’t correlate this phenomenon with anything, except it occurs at an interval of about one month ( merely happenstance, I expect). It is not associated with particularly high or low temperatures. The only way I know to fix the problem is to Disconnect in MaxIm, and cycle the power on the camera.

y standard procedure for an observing session is to power up the camera, Connect in MaxIm, and immediately take a short (e.g. 0.02 sec, to avoid saturating if the sky is still bright) 2x2 binned light frame. If it shows “the problem”, I take another at 1x1, then Disconnect and cycle the power etc. So far, this has always fixed the problem.

Attached are images taken at session startup, named by date and binning factor. The final two from Nov 5/6 are attached to a subsequent reply due to number of files limitation.

On Oct 7/8, the “bad” images show histograms that are “fine-toothed combs” on both 2x2 and 1x1 images. There are three each of those (“startup” 1 thru 6), and all the 1x1 are similar as are the 2x2. Then I rebooted the camera and took images 7 and 8, which are “normal” – a “continuous” histogram, and note the different ADU range even though exposures were similar.

On Nov 5/6, the “bad” images have histograms that are “wide-toothed combs”. I only shot one each of 2x2 and 1x1 (“test” 1 and 2) before rebooting and shooting normal images “test” 3 and 4.

On each of those days, all the images were shot in a fairly short length of time with similar light conditions.

Following is a description of another bootup problem.

I’m debating to start another thread, but this still comes under the category of camera+filterwheel bootup problems, so I’ll put it here: on two occasisions, when taking that first image after powerup and MaxIm Connect, when I press Start to start the exposure, the status message changes to “Waiting for filter wheel” – and never changes. I.e. it doesn’t just show the message for a few seconds and then start the exposure. In one of the instances, I walked out to the observatory (I operate remotely, from the house) – there was no “filter wheel sound” that I hear when first powering up the camera, so I don’t think the wheel was rotating. This filterwheel problem I can “fix” by Stopping the exposure, Disconnecting from Maxim, and then re-Connecting (no cycling of the power to the camera). Simply Stopping the exposure and then Starting it again is not enough: I must Disconnect and re-Connect the camera from MaxIm. But it is not necessary to cycle the power to the camera+wheel. This did not occur on the same powerups as the camera problem documented above and previously, and may not be causally related, i.e. might a separate defect. On one occasion, Oct 01/02, the camera was freshly powered up, but the computer and MaxIm had been running for some days. On the second time, Oct 05/06, the camera was freshly powered up and the computer and MaxIm had been restarted that day. I have no diagnostic data to document this because I don’t know what such data might be (a video of the filterwheel status message not changing?), and also (when investigating another problem that I reported under the MaxIm category) Adam found that MaxIm stops adding to its log file when the filterwheel is enabled – log entries start when the camera is Connected, then stop almost immediately when MaxIm connects the filter wheel.

 

Final two files for Nov 5/6.

 

Did more searching of observing logs, had same filter wheel hang on Sept 10/11 too. Was testing for camera (bad histogram) boot up problems, so I did 8 separate power-ups of the camera: filter wheel hang on one. MaxIm had been restarted shortly before the test session. Required a MaxIm Disconnect/reConnect to fix the problem.

 

Further to this problem -- ignoring filter wheel stuff -- just the AC2020 producing image files with bad histograms):

The problem persists, after camera "repair".

I sent the camera in for repair (Dec 11, 2023). Put it back to work when it was returned (early March 2024)... I assumed the problem would have been repaired, so I just used the camera for normal imaging, i.e. power up at the start of the night, and proceed. Then occasionally had bad boots (see next paragraph). I now have enough "boot ups" on the camera that it is my impression the frequency of bad startups is about the same as before "repair".

Symptoms have not changed: the camera occasionally boots up in a mode that produces images with weird "comb" histograms, i.e. they are not images but just useless arrays of numbers, and the numbers are not random, they cluster around certain values, resulting in a "comb" histogram, as shown in the screen grab posted at the start of this thread.

This persists when I keep taking images, even if I change the binning. The only solution to the bad histograms is to Disconnect, and cycle the power to the camera. As far as I know, the camera functions normally (i.e. properly communicates with the computer etc), except the pixel values are bad.

I do not run an automated observatory, i.e. I power up, put the telescope on target, and take images. Thus I can readily see when my first images are bad. But obviously I could not use this camera in a more automated operation.

Yesterday, I hunkered down and did a test session with 25 boots (power up cycles) of the camera. For all these tests, and all "regular" observing sessions, the camera is connected to the ac power supply it came with, and nothing else is on that power supply. When cycling power I cut the AC power for at least 10 seconds. After power up, I wait at least 10 sec before telling Maxim to Connect. Then I wait at least 10 sec before Starting an exposure.

The result of these 25 boots was that on 2 bootups, the camera booted up in to the mode of producing "images" with garbage histograms. Was using 6.40 and DL Imaging 2.7.1.0 I have purchased the upgrade to Maxim DL 7, but so far "7" is just seeing the guider camera, not the AC2020. I'm trying to figure that out.

 

That didn't fix it.

Now running MaxIm DL v 6.40
AC2020bsi fpga firmware version 4
DLAPI version 2.8.0.0

Just ran 25 camera power/bootup cycles: came up with bad histograms on boots 8 and 10.

g.

 

Hi Gary.

FWIW, I don’t have your camera, or experience of the particular problem, but on reading the thread one question occurs that might have relevance to the problem and has not been explored.

How to you “boot” the camera normally?

1. Camera plugged into the output from an AC - DC power adaptor and camera “boot” is handled by switching off/on the AC power to the adaptor?

or

2. Camera disconnected entirely from an already fully up DC supply and “boot” is handled by connecting the camera power input manually, or via a switch (electronic or manual), to the already fully up DC supply?

Reason for asking is that I have seen intermittent problems with systems that are powered on/off by switching the AC mains supply side with all the equipment already connected to the DC side and the AC-DC adaptor/convertor at initial power-on suffers from either a slow-to-rise to nominal DC voltage output, or, over-shoot of nominal output DC voltage, or, un-suppressed high-voltage spiking occurring on the DC side at initial AC power-on.

…………………
General note:
For any equipment powered by a switch-mode power adaptor it is not recommended to power from on-to-off-to-on again rapidly. If already powered on and then powered fully off a pause of around thirty seconds should be allowed before powering back on.
Poorly designed switch mode power supplies can fail when rapidly power-cycled and produce high voltage spikes on the DC side that can damage whatever they are powering.

 

Thank you for these interesting comments / suggestions, William.

I think I will wait for Cyanogen/SBIG to comment on what is proper procedure. I switch the AC power to the power supply for the camera. When doing many successive tests, as I did yesterday, I (remotely) switch off the AC power, and wait 10 sec before turning it back on. I then wait 10 sec before hitting "Connect" in Maxim DL, and a further 10 sec for Starting an exposure.

Gary Billings

 

Colin, I will naturally cooperate, but will point out the reason I am doing these runs of 25 bootups, is because the problem first presented itself in normal operation, when I do a single "cold" powerup of the camera and the first image is bad. When that happened once, I kind of shrugged it off, but when it recurred, I knew there was a real problem here.

In reviewing all my notes since getting the camera back in March, and last year before the "repair", there MIGHT be a small increase in probability of a bad boot with higher ambient temperatures. Hard to say for sure, given that it happens irregularly, and I have a sampling bias in that I have just done two runs of 25 boots on hot days. I have NOT gone through my archive and extracted heat sink temperatures etc! It is even hard to assess the actual frequency: though those two recent test runs both giving 2 bad in 25 is suggestive of the probability, and feels compatible with the frequency when just doing a single boot per night. Of course, the camera should be booting properly 100% of the time.

 

Regrettably, but I am not surprised, the problem still occurs with longer power off times. I did another 25 power cycles, over 2h45m, with minimum of 60 seconds power off time, sometimes MUCH longer. On cycle 22, it booted with bad histogram.