Hot pixels in dark not removing from light

Discussion in 'Image Capture' started by Kurious George, Aug 10, 2018.

  1. Kurious George

    Kurious George Cyanogen Customer

    Joined:
    May 13, 2015
    Messages:
    22
    Hi folks,

    With my noisier 15-minute exposures, I'm seeing more hot pixels in my lights after calibration. I notice this more on my noisier RGB frames than my L frames. The hot pixels are clearly in the dark master. Attached shows the calibrated R and the master dark (20 frames). All taken at the same temp.

    Under Set Calibration I use...

    Auto-Scale
    Scale Factor 1.0000
    SD Mask, 3, 0.5, Linear, 50%, Ignore black pixels

    Anything I can do to help improve the removal of hot pixels that I see in both the light and master dark?
     

    Attached Files:

  2. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    4,904
    Well it's hard to know for sure without seeing your FITS frames, but this may be normal. The hot pixels are not just brighter, they have more noise. So after dark subtraction half of them will still be a little brighter than the background, and half a little darker.

    It would be very helpful if you could upload FITS frames so I could check it out properly.

    Most people use dithering to get rid of any residual hot pixels. Basically the autoguider shifts the position very slightly from image to image. Then you stack the images, realigning them to get rid of the offset, and use Sigma Clip or SD Mask combining. This obliterates any residual hot (and cold) pixels. Of course MaxIm DL can do all this for you.
     
  3. Kurious George

    Kurious George Cyanogen Customer

    Joined:
    May 13, 2015
    Messages:
    22
    Thanks Doug. I uploaded a master dark and an uncalibrated light. I do dithering but the residual pixels still get in, mainly for my RGB, bot so much L. I started using Kernel Filter > Hot Pixel on the calibrated RGB subs and that fixes the problem. But maybe I could be using better calibrate/combine settings?
     
  4. Kurious George

    Kurious George Cyanogen Customer

    Joined:
    May 13, 2015
    Messages:
    22
    Ah sorry, files too large for upload (32 MB and 64 MB).
     
  5. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    4,904
    Upload to ftp.diffractionlimited.com, incoming folder. Recommend using FileZilla (free download).
     
  6. Kurious George

    Kurious George Cyanogen Customer

    Joined:
    May 13, 2015
    Messages:
    22
    530 Sorry, max 5 users - try again later
    Critical error: Could not connect to server.
     
  7. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    4,904
    I just logged in no problem.

    Might have been busy? Please try again.
     
  8. Kurious George

    Kurious George Cyanogen Customer

    Joined:
    May 13, 2015
    Messages:
    22
    Master dark and uncalibrated light uploaded...

    Master_Dark 7_4096x4096_Bin1x1_Temp-20C_ExpTime900s.fit
    NGC 7331_900sec_1x1_B_frame1.fit
     
  9. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    4,904
    Okay, doing a simple dark subtraction does make a big difference in the level of the hot pixels. For example, there's a hot pixel at coordinates 372, 258 that is 28,375 ADU. That's huge. After dark subtraction it's only 2996. Background level is about 1121, so the height above the background is only 1875.

    Still, that's higher than expected. I don't know your camera's gain (photoelectrons per ADU), but assuming it's 1 then 1 sigma is expected to be around 165... so that value is much too high. It has a 99% chance of being under 500 ADU. Given the other hot pixels still visible, I think it's clear that something is wrong with your images.

    If I use the Auto-Optimize function of Set Calibration, that level goes down to 2064, or in other words something very much in line with what you would expect. Normally I don't recommend using Auto-Optimize unless you have unregulated cooling. This clearly indicates that something is "off" here.

    I notice you applied bias correction to the master dark, but did not supply a bias frame to use when calibrating the light frame. That is more of an issue if you are doing flat-fields but you really need to use bias for both, or not at all.

    At this point all I can say is either the dark frame was not generated properly, or (more likely IMHO) the CCD chip was not at the same temperature for both the light and dark frames.

    It's hard to tell without seeing raw data, instead of a processed master frame, but it could be that your camera wasn't regulating at temperature when the dark frames were taken (there's SET-TEMP setpoint but no actual CCD-TEMP temperature reading in the FITS header so I can't tell for sure). But that's the first thing I would check. The darks and lights MUST be taken at the same temperature or subtraction will not work correctly. I suspect this was not the case here.
     
  10. Kurious George

    Kurious George Cyanogen Customer

    Joined:
    May 13, 2015
    Messages:
    22
    Thanks Doug! This is an FLI PL16803 with 10.0 e- readout noise and 1.43 e-/ADU gain @ 1x1.

    The dark and light were taken at the same CCD temperature (-20C). But I understand the ambient may change the hot pixels a bit.

    Given the situation, do you recommend I apply a Hot Pixel Kernel filter on the subs after calibration but before combining?

    I don't mind sending you the bias master if that will help.
     
  11. Doug

    Doug Staff Member

    Joined:
    Sep 25, 2014
    Messages:
    4,904
    Bias master will not have a significant effect on hot pixels, so I don't think it really matters. It would have a very large effect on flat-fields, or if you were scaling darks to adjust for exposure duration.

    If the ambient has that much effect on the accuracy of the CCD temperature measurement, then the thermistor mounting is poor. It's easy to screw that up by, for example, having the wires too short or by having poor thermal contact.

    Instead of using a filter, I'd suggest using the Auto Optimize feature of Set Calibrate. It definitely knocked the hot pixels down a LOT.

    Aside from that, I heartily recommend dithering.
     

Share This Page