Using Darks

Discussion in 'STF Series CCD Cameras' started by Kenn Hopkins, Nov 25, 2020.

  1. Kenn Hopkins

    Kenn Hopkins Standard User

    Joined:
    Feb 21, 2015
    Messages:
    36
    When ever I use darks I always, without fail, get ugly dark spots in my calibrated images. I use Darks, Flats, Bias, and Lights in CCD Stack v2.95. When I disable using Darks I do not get the dark spots. Within the Calibration Manager I specify a dark master and check the Subtract from flat option using a Bias Master. In the Flat Field tab I select the correct color flat and the same Bias Master. When I use the Darks I'm able to obtain a 'flatter' calibrated image but get those nasty black spots. I do dither my images and some have said not to use the Darks, but when I don't use the Darks the calibrated image is not as 'flat' as when I use Dark Master.

    Any suggestions why the dark spots appear?

    See attached as an example. Art LUM 20201118 scaled scaled.jpg

    Kenn
     
  2. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    9,932
    Please post one unprocessed, raw FITS dark frame, and a raw, unprocessed FITS light frame.
     
  3. Kenn Hopkins

    Kenn Hopkins Standard User

    Joined:
    Feb 21, 2015
    Messages:
    36
    As requested one unprocessed Dark 'D_2020-07-23_19-53-20_Darks_300s' and one unprocessed light 'L_BLUE_2020-11-17_19-57-47_M-33 C-8 HD_180s'
     

    Attached Files:

  4. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    9,932
    That Blue frame has been processed.

    HISTORY = CALIBRATION: Dark Subtract DARK Flat Master - BLUE
    HISTORY = CALIBRATION: Flat Field Flat Master - BLUE
     
  5. Kenn Hopkins

    Kenn Hopkins Standard User

    Joined:
    Feb 21, 2015
    Messages:
    36
    Sorry, I do not understand. I was under the impression you did not want the Master Dark, nor the Flat Master Blue. No images were attached with your reply.

    Kenn
     
  6. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    9,932
    I asked for an unprocessed light frame. That file has been processed. It was calibrated.

    There were zero pixels in that calibrated image, so I think something went wrong with your processing.

    Please upload an unprocessed light frame.
     
  7. Kenn Hopkins

    Kenn Hopkins Standard User

    Joined:
    Feb 21, 2015
    Messages:
    36
    Sorry I did not realize it had any processing completed. OK I can do that. Also attached in the latest Dark.

    Kenn
     

    Attached Files:

    Last edited: Nov 27, 2020
  8. Kenn Hopkins

    Kenn Hopkins Standard User

    Joined:
    Feb 21, 2015
    Messages:
    36
    Doug, back in Apr 21, 2019 you suggested "use a sigma clip or SD Mask process to combine the frames" and that I should dither the images. Through Astro Photography Tool (APT) and PHD2 I dither my images. My processing steps in CCDStack: Reject Hot Pixels ( Strength=10 ) then interpolate rejected pixels ( width(std) 1 iterations 5 ). Reject Cold Pixel with same interpolate reject pixels, Stack using the CCDIS with High Precision then Apply Bicubic B-Spline, and Normalize. The a final Data Reject using Poisson sigma reject ( top images %1 and iteration limit 2 ), finally Stack Combine Mean to finish with one stacked image.
    Kenn
     
    Last edited: Nov 27, 2020
  9. Kenn Hopkins

    Kenn Hopkins Standard User

    Joined:
    Feb 21, 2015
    Messages:
    36
    Of course the previous mentioned process is after Calibration using Master Darks and Master Bias in the Data Subtract tab, Master Flat(based on filter) and Master Bias in the Flat Field tab.
    Kenn
     
  10. William B

    William B Cyanogen Customer

    Joined:
    Jan 8, 2015
    Messages:
    548
    Location:
    Christchurch, Dorset UK
    Hi Kenn.

    Doug will reply soon with some observations and suggestions, FWIW, I had a look at your latest Ha and dark and it appears that your dark has been contaminated by quite a few charged particle hits.

    When you use a master dark created using an average combine method and only a few dark subs are combined into the master dark then those charged particle events assume a large significance when calibrating the lights and this may be the source of your "dark spots".

    Since this problem has been around for a while maybe you are operating in an area with a high background natural radiation, or are using a telescope containing old glass that has a high natural uranium content, or you are situated at a high altitude?

    Whatever the reason, when you build the dark master follow the old "Image times five" rule as suggested by Richard Berry & James Burnell in their book "The Handbook of Astronomical Image Processing"

    (ISBN 0-943396-67-0, 1st edition and ISBN 978-1-942675-08-2 2nd edition, sadly both currently unavailable while Willmann-Bell try to find a publisher to take over the business, but you can still find used copies around)

    Their suggestion to follow the rule " Image times five" for the dark master simply means if you take five "lights" subs to combine into a single image then use twenty-five dark subs to create the dark master.

    When you take the darks make sure that the telescope is pointing to the horizon and not straight up at the sky to minimise the chance of collecting charged-particle hits straight into the sensor.

    Even when the "Image times five" rule is applied there will still be a trace of cosmic-ray / charged-particle hits that make it through to the master dark and though the dark spots in the calibrated image will no longer be black, (they will be grey or coloured spots in an LRGB combined image), you may still be able to detect them in very bright and very dark areas of the calibrated image.

    If this is the case you can either increase the number of darks significantly or use a sigma-reject combine method instead of a straight average combine when creating the master dark.

    Lastly, use a large mount dither when taking the lights, if the average cosmic-ray/charged particle event covers half a dozen pixels and you are seeing a lot of these events then it follows that you must dither the mount more than six pixels to average these out of the final combined image.

    Obviously there is a sensible limit to the number of dark subs that go to make up the master dark and the "Image times five" rule does not have to be followed to ridiculous extremes. For example when combining thirty lights into an image you don't need 30x5 darks (150!) but take a sensible number.
    When I build dark masters for my QSI 683 camera (KAF8300 sensor) I generally use around 60 subs to combine into a master dark though I seldom see charged particle events in my dark subs operating from an observatory at close to sea level.

    My opinion is that if you significantly increase the number of darks subs that go to create the dark master, at least five times the number of lights that you are combining, use a sigma reject method when combining the darks and use a large mount dither then the "dark spots" will disappear.

    HTH

    William.
     
  11. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    9,932
    I agree with what William just said.

    Yes there are a lot of cosmic ray strikes on your image. There are also some hot pixels that are fully saturated. Those are going to make dark and bright spots on your images after calibration.

    You're doing a lot of processing steps that are apparently not being effective. I wouldn't do any of those "reject" functions at all.

    You are dithering, which is good. (It's making your dark spots grow... but that's just because your processing isn't effective.)

    Recommendations:
    1. Make sure your cooler is regulating well... setpoint should be as cold as possible but not exceeding 90% cooler power.
    2. As William suggested take a lot of dark frames
    3. Also take as many light frames as is practical, being sure to dither
    4. Dark frame stacking should be done using Sigma Clip. I've not used your software so I can't give details on the process or settings.
    5. Dark subtract all your light frames.
    6. Align and stack using Sigma Clip. Do NOT do any other processing first. The pixel suppression you are using is probably reducing the effectiveness of the Sigma Clip.
    7. If you don't get good results, go back to Step 6 and adjust your Sigma Clip settings and try again
    If you have trouble with any of the above steps, please ask for assistance from your software vendor. There's nothing fundamentally wrong with your camera data; this is all on the processing side.

    Doug
     
  12. Kenn Hopkins

    Kenn Hopkins Standard User

    Joined:
    Feb 21, 2015
    Messages:
    36
    I really appreciate both of your suggestions, Doug and William. I’m using the software provided with the package of software I received when I purchased the SBIG STF 8300m camera; CCDSoft and CCDStack. I then followed the process suggested by Adam Block in his two DVDs on CCDSoft and CCDStack. I don’t call myself cheap but frugal.

    If I’m to understand Darks correctly I’m supposed to take a minimum of 20 darks using the same time as my Light exposures; a 5-minute Light means I need a 5-minute Dark; which would mean I would have to take 100 minutes to create a Master Dark. I’ve recently hears of Dark Flats using the time interval of the Flats; will these work for creating Master Darks?

    When I take Darks, I have been removing the camera from the scope and just covering it with the lens cap for the nose piece that goes into the focuser. Since I take these Darks at night, I would assume no light falls on the camera chip.
    I also was told Darks do not need to be taken every night, same as Bias not needing to be taken every night. But Flats should be taken every night.

    I’m not sure what CCDStack does when it creates the Master Darks, it has a separate option for that along with a different option for Bias frames. CCDStack has three options for creating a Master Dark, Clip Min/max Mean, simple Mean, and Sigma reject Mean; I will try using the Sigma Reject. I will also try to create a Master Dark using the method I use for creating my Master Light image, it has an option for Standard Sigma Reject or Poisson Sigma Reject.

    Again, I do appreciate both of you guys taking time out of your day to answer my queries.
    Kenn
     
  13. Colin Haig

    Colin Haig Staff Member

    Joined:
    Oct 27, 2014
    Messages:
    7,379
    Location:
    Earth
    Flats have to be taken any time you change the orientation of the camera or reconfigure the optics. So if you remove it at night, then you need to redo the flats. Plus over time, dust accumulates, and so you need to redo flats. If the camera stayed in place, you could do it less often.
    Make sure the darks and bias frames are taken at the same temperatures - they need to be in sets with the lights and flats.
     
  14. William B

    William B Cyanogen Customer

    Joined:
    Jan 8, 2015
    Messages:
    548
    Location:
    Christchurch, Dorset UK
    Kenn.

    “Flat Darks” are optional and you should not use them as a substitute for “regular Darks” that match the exposure time of the lights.

    Some (most) post processing applications will apply scaling to your regular dark frame before it is used to calibrate a flat. This scaling is purely a time based function.

    If your regular dark frame was 100 seconds duration and your flat exposure time was 10 seconds then your calibration software would divide the dark frame pixel values according to the ratio 100/10 and use the resulting “scaled” values to calibrate the flat.

    Scaling works quite well when scaling down a long exposure time dark for a short exposure time flat because any errors in the long dark are reduced along with the wanted dark current measurement.
    But scaling works against you when you “upscale” a short duration dark and try to calibrate a long duration light. Uncertainties, random errors and quantisation errors are all magnified when you “upscale” a dark frame.

    The use of flat darks in addition to regular darks has become more relevant recently with the spread of some designs of CMOS camera sensors that exhibit non-linear dark current and therefore “dark scaling” used to calibrate the flats from the regular darks is noticeably innacurate.

    I have tried image set calibration both with and without “flat-darks” with my KAF8300 sensor and except for a few extremely marginal examples I could not detect a difference in the final image and once you start having to take “flat-darks” to match the flat exposure times for each and every individual filter then the hobby becomes less enjoyable and more like work!

    Bottom line, you can use “flat darks” in addition to your regular “lights-matching-darks” to calibrate your flat frames, but not as a substitute for regular darks, and for your regulated-cooled CCD sensor that exhibits a linear relationship between dark current and time probably of no obvious value for improving the final calibrated image.

    William.
     
  15. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    9,932
    I'll add a note to William's post... if you're going to use dark frame scaling, you MUST use a Bias frame group. Otherwise the dark won't scale correctly.

    If you use Darks and Flat-Darks that match your exposure lengths, then you do not need to use Bias frames.

    (And yes, it's okay to use dark frame scaling with the KAF-8300 CCD sensor. Definitely do not do that with CMOS sensors.)
     

Share This Page