How to despeckle in PI?

[SkyHiker]

I have tried several processing options to remove the R G and B pixel-sized artifacts in the image below. In Gimp it is easy, just select "despeckle" and they are gone. If anyone knows how to do this in PI, please let me know. Thanks!

BTW These artifacts don't appear to be hot pixels. I wonder if they were caused by not setting an offset during acquisition. That may flip them to the other side (dark to brightest). I will check that next time.

[Juno16]

No clue!

I'm no expert though. I only have learned by my own experiences (issues).

They do look like hot pixels to me. How did you pre-process? If you used WBPP, are you using any cosmetic correction? If yes and no, you could try that and save the CC template and apply it in WBPP.

Or, you could bring it into Gimp, for Despeckle, then back to PI.

Just some ideas.

[AstroBee]

Sure looks like something Cosmetic Correction during the WBPP step should take care of.

[SkyHiker]

No clue!

I'm no expert though. I only have learned by my own experiences (issues).

They do look like hot pixels to me. How did you pre-process? If you used WBPP, are you using any cosmetic correction? If yes and no, you could try that and save the CC template and apply it in WBPP.

Or, you could bring it into Gimp, for Despeckle, then back to PI.

Just some ideas.

Thanks Jim and Greg,

I tried automatic cosmetic correction with various parameter values but it didn't work. I can't try CC with master dark because my camera doesn't need darks. I will just use a Gimp excursion for now.

[Juno16]

Of course, do what works for you.

Why do you you feel that your camera doesn't need darks?
I have a ASI533MC (OSC) that has zero amp glow, but if I dig into a stretched dark, I see tons of hot pixels.

My dark library is 2 1/2 years old (yes, I am lazy), but it still does the trick and I have not had any issues with hot pixels (since employing CC).

Here is one of my stretched darks.

Here is the same dark zoomed in.

[AstroBee]

EVERY camera can benefit from darks.

[SkyHiker]

Of course, do what works for you.

Why do you you feel that your camera doesn't need darks?

Because everybody said the ASI2600 does not need darks when it first came out.
There are still lots of people that say you need them but usually they don't have an ASI2600, or they have never done the comparison.
Here is a comparison of 3 images by someone who tried it out.
Elsewhere I read that the histogram for darks up to 600 seconds is below 100, which is negligible.
From my own experience, I have not seen a trace of dark current that I was used to with my Fuji X-a1.

I have a ASI533MC (OSC) that has zero amp glow, but if I dig into a stretched dark, I see tons of hot pixels.

I use bias frames to remove the hot pixels. That should suffice I should think.

[SkyHiker]

Getting back to the hot pixels vs. speckles, I took a preview of the master bias (left), master light with DBE (middle) and master light without DBE (right). I don't see any correlation so these speckles are not hot pixels. They are all auto-stretched.

If I don't stretch the master bias, it is black. I hope that is a valid check that my camera has zero hot pixels, which is much more than I wanted to believe. I checked the maximum value, it is 0.02.

To figure out how to do a histogram in PI I need to get another PhD degree. I just loaded it in Gimp, did a histogram in a log scale, and it is true - all peaks are below 0.02.

[AstroBee]

Not sure what you are looking at or how you are coming to that conclusion but there's no such thing as a camera with no hot pixels. If there were, it would cost $10's of thousands of dollars.

[Juno16]

Just try to pass on my experience with Cosmetic Correction.

I had really good success with WBPP until one dataset. I had hot pixels scattered throughout the stretched masterlight.

I submitted a thread to the PI forum.
https://pixinsight.com/forum/index.php? ... sue.16560/

Take care and good luck!

[SkyHiker]

Just try to pass on my experience with Cosmetic Correction.

I had really good success with WBPP until one dataset. I had hot pixels scattered throughout the stretched masterlight.

I submitted a thread to the PI forum.
https://pixinsight.com/forum/index.php? ... sue.16560/

Take care and good luck!

Very interesting! I went straight back to DSS and found the same area as in the PI preview. I see the same speckles in DSS! This is different than what your PI thread is about I think, because in your case DSS did not produce the speckles. They are fatter just like the stars, but that may be because I stretched the heck out of it.

dssSpeckle.jpg

Because DSS and PI show very consistent speckles, it is unlikely to be caused by the processing. It must be in the raw data. I wonder if I add an offset next time, they will go away. Otherwise, if the bias master is not exact, it may subtract to a negative value, which is a very high value in fixed point.

[SkyHiker]

Here's the DSS section that somehow didn't make it in my last message.

dssSpeckle.jpg (26.89 KiB) Viewed 1452 times

[SkyHiker]

Not sure what you are looking at or how you are coming to that conclusion but there's no such thing as a camera with no hot pixels. If there were, it would cost $10's of thousands of dollars.

From old DSLR bias images that I recall (I delete all my junk files so I can't check), the master bias usually has many hot pixels scattered across the sensor that were bright R, G or B. I don't see that in the master bias above. However, the master bias in PI is not exactly zero, there are pixels with a value up to 0.02 on a scale from 0 to 1, so no bright R, G and Bs. Maybe the ASI2600 has a different kind of hot pixels. Let me know if I am looking at it the wrong way.

I take my bias files in Ekos with the telescope cover closed, however usually on the next morning - not in the dark - at the shortest exposure that I can set. I always assumed that zero external light would be photographed, and I still presume that, but I don't know for sure.

[Juno16]

Just try to pass on my experience with Cosmetic Correction.

I had really good success with WBPP until one dataset. I had hot pixels scattered throughout the stretched masterlight.

I submitted a thread to the PI forum.
https://pixinsight.com/forum/index.php? ... sue.16560/

Take care and good luck!

Very interesting! I went straight back to DSS and found the same area as in the PI preview. I see the same speckles in DSS! This is different than what your PI thread is about I think, because in your case DSS did not produce the speckles. They are fatter just like the stars, but that may be because I stretched the heck out of it.

dssSpeckle.jpg

Because DSS and PI show very consistent speckles, it is unlikely to be caused by the processing. It must be in the raw data. I wonder if I add an offset next time, they will go away. Otherwise, if the bias master is not exact, it may subtract to a negative value, which is a very high value in fixed point.

Is CC enabled in DSS?

You might have mentioned earlier, but what offset are you using?
I’m using 30 on my ASI533 MC (I believe that 50 is default) and I still have 1200 adu of headroom.

Since you have an affected dataset, why not try shooting some darks and use CC in WBPP. Hopefully your data can be salvaged.

Much easier than re-shooting the dataset.

[AstroBee]

Henk, most folks shooting with modern CMOS cameras are not even shooting BIAS frames. They are shooting DarkFlats. That's what I do with my 2600MM and 2600MC. And I can tell you for sure that both cameras have hot pixels.
DarkFlats are shot at whatever exposure your flats are shot at. I use a dimmable ASCOM controllable light panel so I shoot all of my LRGB flats at 1 sec and my narrowband flats at 3 sec. That way I can shoot just one set of DarkFlats for broadband and one set for narrowband.

[SkyHiker]

Just try to pass on my experience with Cosmetic Correction.

I had really good success with WBPP until one dataset. I had hot pixels scattered throughout the stretched masterlight.

I submitted a thread to the PI forum.
https://pixinsight.com/forum/index.php? ... sue.16560/

Take care and good luck!

Very interesting! I went straight back to DSS and found the same area as in the PI preview. I see the same speckles in DSS! This is different than what your PI thread is about I think, because in your case DSS did not produce the speckles. They are fatter just like the stars, but that may be because I stretched the heck out of it.

dssSpeckle.jpg

Because DSS and PI show very consistent speckles, it is unlikely to be caused by the processing. It must be in the raw data. I wonder if I add an offset next time, they will go away. Otherwise, if the bias master is not exact, it may subtract to a negative value, which is a very high value in fixed point.

Is CC enabled in DSS?

DSS does not have that CC as far as I know.

You might have mentioned earlier, but what offset are you using?
I’m using 30 on my ASI533 MC (I believe that 50 is default) and I still have 1200 adu of headroom.

I mentioned that right away as the possible source of the problem. I had not set any.

Since you have an affected dataset, why not try shooting some darks and use CC in WBPP. Hopefully your data can be salvaged.

Much easier than re-shooting the dataset.

That's not a problem at all, Gimp easily removed the speckles. The data set is only an hour's worth of the Western Veil, I attached it here. I discovered that I still have some coma left and ordered another ring set then I will retake it.

[SkyHiker]

Henk, most folks shooting with modern CMOS cameras are not even shooting BIAS frames. They are shooting DarkFlats. That's what I do with my 2600MM and 2600MC. And I can tell you for sure that both cameras have hot pixels.
DarkFlats are shot at whatever exposure your flats are shot at. I use a dimmable ASCOM controllable light panel so I shoot all of my LRGB flats at 1 sec and my narrowband flats at 3 sec. That way I can shoot just one set of DarkFlats for broadband and one set for narrowband.

I'm curious how much difference it makes. The link I provided has three images as evidence that taking darks and dark flats makes no difference at all for the ASI2600. I don't know if you have tried this yourself. I may try it to see if it makes a difference once I get to the point where it is useful. Based on the evidence, I doubt if it will. For now, I have gradients from streetlights and fog that are much more of a concern. It's off topic, something for a new thread.

[Juno16]

Possibly the driver set a default offset?
Does the FITS header display a “0” offset?

I also use darkflats with my camera. I just use an inexpensive light panel (or my ipad) for my flats and manually adjust exposure in the 1-5 second range to aim for 50% max adu for my sensor. Cap it and run darkflats (in a dark room or outside at night).

[AstroBee]

You should do a poll with 4 choices.
1. No Darks, Bias, DarkFlats
2. Darks only
3. Darks and Bias
4. Darks and Dark Flats.

I'm putting my money on #4 by a landslide.
Everybody else can't be wrong.

[DEnc]

...I submitted a thread to the PI forum.
https://pixinsight.com/forum/index.php? ... sue.16560/

Thank you for this! Post #20 was very interesting.

Off topic, now that I'm under heavily clouded skies of northern coastal California I've pivoted from astrophotography with a monochrome camera to daytime DSLR imaging in raw format. White balance and demosiacing have been challenging. The comments about VNG versus bilinear interpolation were very welcome.