Resolved All Sky 340M crashing ~ daily.

Ok time for an update... I finally got up to my observatory and retrieved the camera. It's been sitting and running on my bench for over a week without fault. Same power supply, same USB adapter. The only differences are the computer (home machine vs EAGLE at the dome) and the serial cable (10m @ the dome, and running underground, so I didn't pull it out). I'm at a bit of a loss about what to try next - thoughts?

 

- same length of cable at home

Yup - I'll order another 10m serial cable to test this I think.

- home computer to the observatory

Yup - Another big job / trips. But yes, I can certainly try this and rule out something with the eagle.

- try setting a lower baud rate on the camera when it is at the observatory. Images will download slower, but it may work more reliably.

Have already done this, no improvement.

- is that 10m of cable 18AWG and shielded or what?

Heavier duty than that, it's double insulated, but not shielded. It runs through a big conduit with cables for the bolt wood etc also.

- a possibility is that the USB to RS232 adapter you are using doesnt have high enough drive voltage to send commands to the camera over 10m of cable. eg real serial ports supply +12/-12V on their data pins; the USB to serial adapters are usually +5/-5V or lower. You could put a voltmeter on the TX pin and GND on the serial adapter and see what it is.

Using the adaptor suggested by SBIG but agree it could be an issue.

- a less likely possibility is that the USB to RS232 adapter isn't getting a full +5V from the USB of the Eagle. Try adding a POWERED USB Hub (eg has an AC wall plug adapter, or a StarTech industrial hub running off +12V).

- reload the Eagle fresh, OS and all.
BTW Which Eagle is it?

Current version 3 - In retrospect not something I would buy again however, because doing a full clean reinstall isn't possible. Also I'm not about to mess up everything else that's working perfectly because this camera is flakey...

Beyond that, no idea.
Good luck,
Colin

The thing that still doesn't make sense is that it takes a power cycle to get it going again. It's as if the camera itself is crashing somehow ... if it was just a flakey serial connection that doesn't make sense ... I still wonder if it's got a hardware fault, dry joint etc... and moving the camera jiggles it just right etc.

 

James and SBIG,
Was there a resolution to this problem. We are having a similar problem with our all-sky 340. I am just curious why doesn't anyone make an all-sky IP cam that you can just hook into a network

 

I added a huge ground rod for the camera which improved things slightly but I've never got it stable at my dome. I am literally right now rebuilding my setup onto a new Dome computer. I can't replicate the fault at home - so perhaps there is something about the dome computer's USB setup? It's really frustrating, this is not a cheap camera, and it is 10+ year old technology with zero updates for years... it should be bullet proof. There is so much about astro imaging which is stupidly clunky or just really old software and hardware... the curse of a small market.

 

Yup - I've tried all those things... and so much more. I've ultimately replaced my dome computer as a last resort. The camera has been totally reliable on the bench at home, and infuriating in the field! At this stage I am hopefully that the Eagle has a crappy USB chipset, or some other quirk which has been the cause of so much frustration. An expensive exercise.

Re a new camera - Very pleased to hear you are considering a modern iteration. From my perspective...

The primary use is to asses sky conditions remotely to check and second guess the weather station.
A full sky view (or very close too) is essential.
Mono is fine, thought a good colour camera would be fine also.
More resolution would be nice, but not at the cost of sensitivity.
One image every 5 mins is fine.
A cooled chip, which allowed for lower noise, and practical usage for things like chasing meteorite showers etc would be nice... I can appreciate the potential impact on longevity though for an always on device.

I can really appreciate the complexities of wired vs wireless and a standalone or tethered unit.
I'd have thought ethernet was a pretty universal standard. If users need longer distances then there are simple ways to extend (fibre, wireless bridges etc)

The longevity issue is true for nearly all good astro gear, reinstalling a new dome computer has been a painful reminder of how many quirks and hacks exist to get around old hardware, old protocols on new operating systems...

 

Hi James & Bradley.

James, did you ever try a galvanic isolator on the RS232 port (optical isolator) to eliminate the ground loop possibility, as discussed earlier in the thread (and in the All Sky manual) ?

With a lifetime experience of installing and maintaining RS232 comm links in medical systems the commonest RS232 failures found during commissioning and during equipment lifetime maintenance are ground loops and incorrect cabling.

Is the cable the correct specification for RS232 comms and has it been correctly terminated and installed?

For reliable RS232 comms above 9600 baud, up to 115,200 baud, with cable runs over a couple of meters the cable must be low capacitance, low impedance and the Rx -Tx cores should be twisted-pairs with an overall external shield. For baud rates above 115,200 the Rx -Tx cores should be twisted-pairs with individual shields and an overall cable shield.

In a correctly installed RS232 cable, the outer shield is connected to chassis ground only and not 0v (chassis ground and 0v are only allowed to be connected at a single point, usually in the power supply, and if the host computer and connected equipment are on different power supply networks the shield is connected to chassis ground at one end only and isolated at the other, if on the same power supply network the shield is connected to chassis ground at both ends.
Where the host end and the client end are on different power supply networks a galvanic (optical) isolator on the RS232 cable is essential.
The Rx and Tx twisted pairs are Rx + 0v and Tx +0v with the 0v’s of both Rx and Tx connected to signal 0v at both ends.
The commonly held belief that only three cores are required, Rx, Tx and 0v over a parallel-cored cable is often quoted but is vulnerable to interference both from external sources and internally via cross-talk between the Rx and Tx signals and the problems increase with the length of the cable run.

The commonest cabling errors are using unshielded and untwisted-pair (parallel-core) cabling, or using twisted-pair but where the Rx and Tx have been twisted together instead of Rx + 0v and Tx + 0v.

The correct specification Rs232 cable (shielded w’ twisted pair cores) can be expensive. In the medical environment I am familiar with a four-core, twin twisted pair, single shielded, low-smoke toxicity RS232 cable is around €2-€3 per meter however for an observatory a suitable low-cost alternative for baud rates up to 115,200 is regular CAT5 STP cable (STP = Shielded Twisted Pair).
If using CAT5 STP cable the unused cores must be connected to signal 0v at both ends and not left floating.

In the systems I worked on, low frequency, sporadic, RS232 comms errors with a frequency extending over several hours or days tended to be ground-loop induced. As the load on the building power supply changes with usage and weather so the potential difference between mains neutral and ground varies, leading to variable ground-loop currents flowing along the RS232 cable. In some cases cable-duct fires were the result, and is the reason we always specified galvanic isolators for RS232 equipment interconnections outside of our own local domain. High frequency errors, occurring over minutes, tended to be the wrong specification of cable, or incorrectly terminated.

If your system works reliably on the bench but not when installed then an impermissible ground loop would be the first place to look but if you are using an unshielded and un-twisted core cable that is running alongside other cables, particularly cables carrying mains AC and/or in metal ducting, then the cable specification can also be the reason.

Start with the correct type of cable, properly installed and terminated, screened twisted-pair and unused cores tied to signal 0v, screen tied to chassis ground at one end only, if the problem remains install a galvanic isolator.

A test of the RS232 Rx and Tx signals at host and client ends of the cable with an oscilloscope will show pretty much instantly if signal smearing, cross talk or mains frequency AC superimposition are present, if you have access to an oscilloscope or know anyone that can check for you it will save a lot of head scratching.

HTH

William.

 

James, did you ever try a galvanic isolator on the RS232 port (optical isolator) to eliminate the ground loop possibility, as discussed earlier in the thread (and in the All Sky manual) ?

Yes ... I have used a number of different USB convertors, currently a StarTech Opto Isolator. Doesn't improve my issue.

With a lifetime experience of installing and maintaining RS232 comm links in medical systems the commonest RS232 failures found during commissioning and during equipment lifetime maintenance are ground loops and incorrect cabling.

Is the cable the correct specification for RS232 comms and has it been correctly terminated and installed?

I believe so, I have tried a number of different commercial cables. No difference between them.

For reliable RS232 comms above 9600 baud, up to 115,200 baud, with cable runs over a couple of meters the cable must be low capacitance, low impedance and the Rx -Tx cores should be twisted-pairs with an overall external shield. For baud rates above 115,200 the Rx -Tx cores should be twisted-pairs with individual shields and an overall cable shield.

In a correctly installed RS232 cable, the outer shield is connected to chassis ground only and not 0v (chassis ground and 0v are only allowed to be connected at a single point, usually in the power supply, and if the host computer and connected equipment are on different power supply networks the shield is connected to chassis ground at one end only and isolated at the other, if on the same power supply network the shield is connected to chassis ground at both ends.

Where the host end and the client end are on different power supply networks a galvanic (optical) isolator on the RS232 cable is essential.
The Rx and Tx twisted pairs are Rx + 0v and Tx +0v with the 0v’s of both Rx and Tx connected to signal 0v at both ends.
The commonly held belief that only three cores are required, Rx, Tx and 0v over a parallel-cored cable is often quoted but is vulnerable to interference both from external sources and internally via cross-talk between the Rx and Tx signals and the problems increase with the length of the cable run.

The commonest cabling errors are using unshielded and untwisted-pair (parallel-core) cabling, or using twisted-pair but where the Rx and Tx have been twisted together instead of Rx + 0v and Tx + 0v.

The correct specification Rs232 cable (shielded w’ twisted pair cores) can be expensive. In the medical environment I am familiar with a four-core, twin twisted pair, single shielded, low-smoke toxicity RS232 cable is around €2-€3 per meter however for an observatory a suitable low-cost alternative for baud rates up to 115,200 is regular CAT5 STP cable (STP = Shielded Twisted Pair).
If using CAT5 STP cable the unused cores must be connected to signal 0v at both ends and not left floating.

In the systems I worked on, low frequency, sporadic, RS232 comms errors with a frequency extending over several hours or days tended to be ground-loop induced. As the load on the building power supply changes with usage and weather so the potential difference between mains neutral and ground varies, leading to variable ground-loop currents flowing along the RS232 cable. In some cases cable-duct fires were the result, and is the reason we always specified galvanic isolators for RS232 equipment interconnections outside of our own local domain. High frequency errors, occurring over minutes, tended to be the wrong specification of cable, or incorrectly terminated.

If your system works reliably on the bench but not when installed then an impermissible ground loop would be the first place to look but if you are using an unshielded and un-twisted core cable that is running alongside other cables, particularly cables carrying mains AC and/or in metal ducting, then the cable specification can also be the reason.

The camera is at the other end of ~10m of serial cable which runs along side a number of other cables... Ethernet for an SQM and the serial cable for my Boltwood. Plus 12v for the camera itself. If I still have issues after the computer switch out them I'm prepared to pull the serial cable and swap it over, but it's under ground in a conduit and it will be a huge job.

Start with the correct type of cable, properly installed and terminated, screened twisted-pair and unused cores tied to signal 0v, screen tied to chassis ground at one end only, if the problem remains install a galvanic isolator.

A test of the RS232 Rx and Tx signals at host and client ends of the cable with an oscilloscope will show pretty much instantly if signal smearing, cross talk or mains frequency AC superimposition are present, if you have access to an oscilloscope or know anyone that can check for you it will save a lot of head scratching.

Now that isn't something I had considered to test this... I can probably get my hands on something from an Amateur Radio buddy.

HTH

William.

 

As you have used an opto-isolator you can rule out ground loops, at least on the signal cable.

A couple of other things to try, fit ferrite cable clamps over the RS232 and camera power cables at the host computer and camera ends to reduce the chance of parasitic interference.
(For reliability, ferrite clamps should be on both ends of all low voltage power and signalling cabling in an observatory).

If excess cable is laying around it should not be tidied in continuous coils but tidied in a figure-of-eight pattern.

If the oscilloscope test shows significant signal voltage drop on the RS232 cable and if your host computer has a spare expansion slot then a dedicated RS232 expansion card is a much better option as you will have the full 12v Rx - Tx signal available against just 5v with a USB-RS232 adaptor.

William.

 

One final thought...

Make sure the power jack at the camera end really is 5.5/2.0 and not 5.5/2.1 or even 5.5/2.5, and inspect the plug to make sure the solder tags and jack pins are tightly riveted, had a few spurious faults over the years due to loose riveted solder tags on jack plugs or the wrong sized plugs supplied in wrongly labelled packs.

 

A week with the new dome computer up and running and zero faults. Fingers crossed this might finally be solved. I will indeed install some ferrules - I already have a box of them. I'll update this thread again in another week or so hopefully with the same positive prognosis.

 

And now another week and a half on the camera continues to work without a single fault. What a saga this has been.

In summary regardless of which serial cable, power cable or USB serial convertor the same issue occurred with the EAGLE computer. It crashed the 340 camera such that a power cycle was the only way to recover.

As a final (and rather expensive) step replacing the computer with a FIT PC IPC2 would appear to have finally resolved the problem. Thanks all for your patience, even when I was frustrated or frustrating!