FW8S-Aluma Error - with no error message

Happy Monday Team DL,

I'm seeing an odd issue with my FW8S Aluma filter wheel. It's been in place for a few months now, and operates mostly perfectly. I'm doing time series BVI photometry on high amplitude delta scuti stars, and I'm switching filters every couple of minutes over the course of the night. I'm using N.I.N.A. to control the camera/filter wheel using the ASCOM driver (v6.4.13.0). At a certain point during the night, the software will initiate a filter change.....that never completes receives a completed notification and the software ends up sitting in limbo until the Nautical Dawn trigger causes the session to execute the shutdown sequence. The net result is that I end up with no data for the rest of the night. N.I.N.A. doesn't throw any kind of error message until hours later when it's unable to change the filter for a focus sequence at the end of the night. Here's an excerpt from the N.I.N.A. log that illustrates what I mean.....there's a call to switch filters, and then nothing happens for hours.

2023-05-08T00:35:55.7388|INFO|SequenceItem.cs|Run|206|Starting Category: Filter Wheel, Item: SwitchFilter, Filter: I
2023-05-08T00:35:55.7428|INFO|FilterWheelVM.cs|ChangeFilter|104|Moving to Filter I at Position 4
2023-05-08T00:35:55.7438|INFO|FocuserVM.cs|MoveFocuserInternal|205|Moving Focuser to position 16417
2023-05-08T04:25:56.1783|INFO|TimeCondition.cs|Check|245|TimeCondition finished.
2023-05-08T04:25:56.1793|INFO|TimeCondition.cs|InterruptWhenTimeIsUp|62|Time limit exceeded - Interrupting current Instruction Set
2023-05-08T04:25:56.1823|INFO|SequenceItem.cs|Run|224|Finishing Category: * Instruction Set *, Item: NINA.Sequencer.Container.SequentialContainer, Strategy: SequentialStrategy, Items: 2, Conditions: Condition: TimeCondition, Time: 4:25:56h, Triggers:
2023-05-08T04:25:56.1823|INFO|SequenceItem.cs|Run|206|Starting Category: * Instruction Set *, Item: NINA.Sequencer.Container.SequentialContainer, Strategy: SequentialStrategy, Items: 2, Conditions: Triggers:
2023-05-08T04:25:56.1823|INFO|SequenceItem.cs|Run|206|Starting Category: Telescope, Item: SlewScopeToAltAz, Coordinates: NINA.Astrometry.InputTopocentricCoordinates
2023-05-08T04:25:56.1823|INFO|TelescopeVM.cs|SlewToCoordinatesAsync|904|Slewing from RA: 16:31:20; Dec: 16° 51' 55"; Epoch: JNOW to RA: 19:09:03; Dec: 21° 56' 28"; Epoch: JNOW
2023-05-08T04:26:09.6236|INFO|SequenceItem.cs|Run|224|Finishing Category: Telescope, Item: SlewScopeToAltAz, Coordinates: NINA.Astrometry.InputTopocentricCoordinates
2023-05-08T04:26:09.6236|INFO|SequenceItem.cs|Run|206|Starting Category: Focuser, Item: RunAutofocus
2023-05-08T04:26:09.7961|INFO|FilterWheelVM.cs|ChangeFilter|104|Moving to Filter C at Position 0
2023-05-08T04:26:09.7961|INFO|FocuserVM.cs|MoveFocuserInternal|205|Moving Focuser to position 15746
2023-05-08T04:36:09.8237|ERROR|AutoFocusEngine.cs|SetAutofocusFilter|1201|Failed to change filter during AutoFocus



I'll be down at the observatory sometime in the next week or two to add a couple of new filters and plan to swap out the USB cable in that time, though I don't think the USB cable is the issue. In the meantime, any ideas what could be happening? There is no ASCOM log file or error being generated. There's a call to switch the filter.....and then nothing happens for the rest of the session.

Camera is an Aluma CCD47-10, Firmware revision 27.

Thanks for any assistance or direction you can offer.

Mike

 

Thanks for the quick follow up Colin. I've checked all settings on the PC as suggested in your linked post. Selective Suspend had already been disabled, but the device manager settings needed to be updated. I'm still skeptical that this is the cause....as the PC has just communicated with the camera over the same USB cable at the point this occurs, but it can't hurt to update those settings. I got through the full sequence last night without reproducing the issue, so that's good. I have all of my devices (Mount, Focuser, Camera & Flat Panel) all plugged into the same USB 2.0 hub on the observatory PC. When I get down there next (hopefully in the next few days), I'll plug the camera into the USB 3 hub that's also on the observatory PC. This will isolate it from the other devices. I'll keep you posted, and I'd be willing to put the new DL Imaging drivers to the test when they're available.

Thanks,

Mike

 

The cause has been the same every time.....but even that is intermittent. It's happened five times now, but that's over about 20 days of observing.

Slew to variable star 1.....all 20 second exposures here.
3 exposures in B
3 exposures in V
3 exposures in I

Slew to variable star 2....all 30 second exposures here.
3 exposures in B
3 exposures in V
3 exposures in I

....repeat until nautical dawn, about 4:30am. For what it's worth, the two stars are close together...maybe 5 degrees apart (DY Her and V1116 Her).

At some point during a filter change, things just "stop happening". I'll have a period of 3 to 4 hours where nothing else happens, presumably because the filter change never finishes.
Since I'm in bed and not monitoring, I'm not aware until the next morning.

At nautical dawn, I have a trigger that interrupts the sequence and does a final focus of the telescope before shutting down - measuring out how much focus changes over the course of the night and the temperature drop.

That focus sequence will try to switch to the clear filter and triggers an error 10 minutes later when that filter change times out.

At this point, I'm unable to do anything to/with the camera and filter wheel until I power cycle the camera.
Other than a few hours of lost data it doesn't appear that there are any other issues going on. N.I.N.A still functions normally, telescope gets parked, roof closes, etc.
There's just a call to switch filters over the ASCOM driver that never finishes and it doesn't trigger any errors that I could trap some other way.

So...to answer your question, I'm not sure how to do a test run. Because it's intermittent, I'm not sure what combination of events causes it to happen.

 

Hey Team DL,

Any update on the availability of new DL Imaging ASCOM drivers? I had the filter wheel hangup occur again last night at 10:45.....and lost the rest of the night's observing plan as a result. The last instruction in the N.I.N.A. log file is:

2023-05-25T22:45:13.4907|INFO|SequenceItem.cs|Run|206|Starting Category: Filter Wheel, Item: SwitchFilter, Filter: B
2023-05-25T22:45:13.4917|INFO|FilterWheelVM.cs|ChangeFilter|104|Moving to Filter B at Position 1

Nothing in the ASCOM folders for any sort of logging...it's like the instruction to the filter wheel disappears into the ether. The camera and filter wheel will stop responding to all commands at this point - including shut down commands or warming up the cooler. The only solution is to cut power to the camera, which I don't really like to do when the CCD is at -30º.

Thanks,

Mike

 

Colin,

Thanks for the usual quick response. Downloading and installing now, and I'll put it to work tonight if I can (clouds may be an issue). If not, tomorrow night. The implementation of additional logging will definitely be helpful if the issue reoccurs.

Thanks again,

Mike

 

Installed, double checked....and sorta tested.

I installed new drivers and then ran a routine similar to my normal sequence taking 3 images of the wall through each filter before making a filter change. I did this for 100 iterations without issue, so hopefully this is a good sign. The real test comes at night with the mount running too, but early results are encouraging. Thanks for all the assistance.

Mike

 

Hey DL Team,

The issue where NINA sends an instruction to change filters.....and then nothing else happens for the rest of the night has occurred twice more since installing the updated drivers. At this point I'm unable to reliably use the filter wheel, so multi-filter imaging/photometry isn't a possibility. Here's the last thing from the N.I.N.A. log file from last night's interrupted run; note that from 2:43 until I interrupted everything to manually begin the shut down at 3:55, nothing happened. The display in N.I.N.A. at the time simply reads "Switching Filter".



At Colin's suggestion, I've enabled logging at level 5 in the dlapi driver to start gathering more information. Here are relevant lines for calls to the filter.


The camera and filter wheel will not function at this point until I power cycle the camera. Troubleshooting the source, I disconnected from N.I.N.A. and connected to the camera and filter wheel from Maxim DL 6.29. I attempted an image using the I filter (which is what should be in the imaging slot at this point) and Maxim DL too is not able to use the hardware. It stays locked in a filter moving/Waiting for filter wheel status:


Attached is also a .zip file containing both the N.I.N.A. log file from the evening and the dlapi log file that covers that time period to assist in troubleshooting. The telescope is located remotely but I can provide access via TeamViewer or some other remote desktop software if it helps to get a direct look at the system.

Thanks,

Mike

 

The beta did not resolve the filter wheel issue.

The filter wheel fails to respond after some amount of time during time series photometry runs.

It's not isolated to N.I.N.A.
It also happens with MaximDL.
I've swapped out the USB cables to ensure it's not the cable.
There are no hubs, extenders or any other shenanigans involved. The USB cable is connected directly to the observatory PC.
I've unplugged all other USB devices from the observatory computer.

Still, the issue occurs.