TheSky X autodark issue suspected

OK, thank you, I am off to CCDStack.

 

OK, CCDStack says pretty much what you all have been saying: increase dithering; increase color subs. So here is an image of M33. It is 15 3min Lums, and 15 each of 1min RGB. Dithering is set at 10. All the images were dark calibrated first, then bloom elimination, then normalized, then sigma data processed, then each filter group median combined, then color combined. This is probably one of the poorest images I have ever produced.
Jim

 

Hi Jim.

Unfortunately no one here uses CCDStack and it is appears that the processing sequence you are following is wrong in some way.

It is impossible to say where the sequence is wrong from a completed image although the image still contains displaced colour spots so apparently sigma reject is not working. In most of the processing packages that I use sigma reject is not a separate function from combination, rather it is a method of combination, i.e, Sigma reject combine, Median Combine, Average Combine etc.

The only way we can see what is going wrong is if you take up Doug's suggestion and provide a complete raw data set for this target and then we can check the data for integrity and assemble it using our own software and provide you with either a reasonable answer for the failure or an acceptable worked image with a process sequence. If we can produce an acceptable image from your raw data then the problem must be in the sequence you are following when using CCDStack.

Without the raw data we can do nothing else.

Ideally, for a data set this size use a free Google Drive account, place all the individual images in a single master folder containing named sub-folders ( Bias, Dark, Flat, Red, Green, Blue, Lum), zip compress the master folder and upload it to Google Drive and post back here with a share link to that master folder, we don't want any pre-processed images here, just the raw data as it comes from the camera.

Note that if you don't have bias frames then we need both three minute darks for the luminance frames and 1 min darks for the RGB frames, we need a minimum of ten darks of each, if you have separate bias frames then supply a minimum of twenty bias frames.

William.

 

Hi William

Let's see if this will work. I have a good friend who has created a Drop Box site just to get you the raw images requested. They are some 50+ images of about 110MB. If you send Chris Scherpenseel your email address, he will send you the images in a Drop Box with instructions to receive them. These are all raw, uncalibrated images.

Thank you very much for your help.

Jim

 

William
Thanks for catching my mistake. Corrected.
Jim

 

Good Morning/Afternoon Willian
The requested files should be coming from Chris's Drop Box within a few hours. I thought you wanted only raw images, so I sent the uncalibrated ones. The calibrated ones are on their way.
I noticed Drop Box scrambled the order of the images. So if they are not identified by filter used, here are the numbers: images 9427-9441 are luminance; 9442-56 red; 9457-9471 green; and 9472-9484 blue. Please note the last few blues should be tossed as the mount crossed the telescope limit and stopped tracking.
The darks 9730-39 are 3min, darks 9740-49 are 1min, bias 9750-9769 are 0min. All at -15C.
Thank you again for your help. I am trying to determine if I have an old camera (CCD chip) problem or a software image manipulation (CCDStack) problem. Your help is most appreciated.
Jim

 

William
In response to your email:
The raw calibration darks and bias requested are with the last pack from Chris.

Jim

 

Hi Jim.

I have processed your raw image set for M33 and you will be pleased to know that there are no issues with the camera, the raw data is normal and the post processed image is normal too.

The calibration frames, bias and darks, look and measure as expected.
There are a few defective partial-columns and a good smattering of both hot pixels and dead pixels but nothing unexpected for this age of camera and model of sensor and nothing that can't be post-processed away.

The previously noted large grouping of hot pixels are not present in these frames and I suspect that they were a post-processing artefact.

Although I use MaxIm for image capture I have not post-processed in MaxIm for many a year and because you don't use MaxIm either I just ran with your data set through PixInsight using a very basic workflow.

I did not think it was appropriate to describe the PixInsight process here, as this is a forum for Diffraction Limited products, but as soon as I have a moment spare I will write down the process sequence used on this image and send that to you using the private message service via this forum, you should be able to adapt that process sequence to CCDStack.

Given the total integration time for this data set the results look normal to me.

The final result would have benefitted from a set of matching flats as your optical system has a noticeable vignetting field and with a target that fills the frame such as this it was difficult to remove that vignetting field using PixInsight processes but I made the best of it I could.

I am sure that the problems you are experiencing are just finding the correct processing steps, in the correct order in CCDStack.

I have attached a 16bit .fit final image to this reply and for others interest I have attached a reduced quality jpeg.

William.

 

William
Thank you so much for your time, expertise and effort. It is a relief to know that the camera, although carrying a few scares of old age, is not the problem. I will go to work with CCDStack and try and determine if it's their software or my operator error. I would appreciate the process sequence you used with PixInsight whenever you have the time.
Thank you again. Your efforts are most appreciated.
Jim

 

Thanks Colin.

I have outlined the sequence of steps used in PixInsight on Jim's image below.

I have sent Jim a PM with attached .pdf listing the steps I followed in greater detail, though how much of that can be applied to CCDStack I do not know.

I tried to use a minimal tool set in PixInsight and non of the advanced techniques or modules available in the hope that Jim might find some commonality with CCDStack. As a result, the image I produced is not the best that could be achieved and there is much more in Jim's raw data than I was able to pull out using the reduced tool set and limited time.

----------------------------------------

Blink Module:
Ran all the raw data subs through Blink and rejected those subs showing any obvious defects.

ImageIntegration Module:
Created Master Bias and Master Darks to match Luminance and Colour subs, combination method = 'Average', 'no normalisation' and 'Winsorized Sigma Clipping' with 'Scale and Zero Offset' normalisation for outlier rejection.

ImageCalibration Module:
Used the master calibration frames created in the last step to calibrate the luminance and colour subs.

CosmeticCorrection Module:
Used CosmeticCorrection Module to identify all remaining hot, warm and dead pixels in all the subs and remove them.

StarAlignment Module:
Selected and made a copy of the best Luminance sub as an alignment reference and co-registered all the Luminance and Colour subs to this one image.

ImageIntegration Module:
Ran each set of channel subs individually, Luminance, R, G, and B, through the ImageIntegration Module to create calibrated and aligned masters.
Used 'Average' as the combination method, 'Additive with Scaling' for the normalisation method, 'Use Noise Evaluation' for the weights method and 'Winsorized Sigma Clipping with 'Scale + Zero Offset normalisation for outlier rejection.

LRGBCombination Module:
Combined the individual R,G and B frames from the last step into a RGB composite but left the luminance channel un-combined.

BackgroundNeutralisation Module:
Selected a small preview of a bland area of background in the RGB image and used that to remove a colour imbalance resulting in a strong green cast, leaving the background a neutral grey and a more natural looking image.

ColorCalibration Module:
Selected a preview of the entire galaxy core to serve as a white reference and a second preview of a bland area of background for a background reference and applied the module to produce a white balanced image where the galaxy and star colours looked natural.

DynamicBackgroundExtraction Module:
Ran DBE on both the Luminance and Colour images to remove an obvious vignetting field that could not be removed during image calibration as there were no flat frames available.

ArcsinhStretch Module:
Ran ArcsinhStretch on both Luminance and Colour images to compress the image range while preventing pixel saturation.

HistogramTransformation Module:
Applied Automatic Screen Stretch to both the Luminance and RGB images and transformed those screen stretch values into a permanent stretch using the HistogramTransformation Module.

LRGBCombination Module:
Combined the separately processed RGB and Luminance images into a single LRGB image.

HDRMultiscaleTransform Module:
Ran MultiScaleTransform to compress the bright galaxy core and better differentiate the gas clouds and bright spiral arms.

ACDNR Module:
Ran ACDNR to reduce chrominance noise that was visible in the background areas of the image.

HistogramTransformation Module:
Ran HistogramTransformation Module to remove unwanted and unused black level and darken the background sky from a mid-grey closer to dark-grey/black but leaving just enough of the fainter parts of the galaxy visible.

DynamicCrop Module:
Cropped away the overlapped edges of the image where dithering and drift revealed a steady displacement of the subs as acquisition progressed during the capture session.

Saved completed image as 16Bit.fit and .jpg.

William.

 

Thank you William,
This process is similar but also quite different than CCDStack. However, I bought a CCDStack tutorial by Adam Block and am going through it now. We'll see what I get. I'll start with a new set of images once the moon is down.
Its Christmas Eve and I want to thank you, Colin and his entire Canadian squad for your help. It is most appreciated.
With kind regards and wishes for a more peaceful 2021.
Jim