Bad Flats or LP?

[Baskevo]

I don't know if anyone can tell me this, but does this look like my flats are over-correcting? Or could this just simply be the heavy light pollution?

[UlteriorModem]

It's a combination of vigenetting made worse by sky glow. I have to deal with this as well.

Since your using Pixinsight I suggest learning about background extraction

[Mac]

I have the same issue when using the Optolong LP filter pointing into skies with heavier LP and the barreling effect does not show in the flats.

I use Photoshop to finish with Dodge and burn to bring down what PixInsight won't correct.

[yobbo89]

i'm on the same path tonight, can't really take new flats atm. so i'm gonna stack the frames without the flats then do a background extraction.maybe give that a shot as well.

[STEVE333]

Nasty problem.

I'm assuming these are the images after correction with the Flats?

LP will generally show up as a gradient being brighter on one side (or top or bottom) than the other.

What you are seeing is vignetting which produces an image that is brighter in the center and darker towards the edges. Of course there is probably some LP mixed in too.

If the Flats are correcting the vignetting properly, then, after calibrating the images you will be left with only the LP gradient. In your case there is still a strong amount of the vignetting present (over or under corrected) even after correction with the flats. I'm having some problems with this too, but, mostly with the Red filter!! Not sure why one filter would have more problems than another. Still learning about this with my new ASI1600MM setup. Always something.

Steve

[Baskevo]

Nasty problem.

I'm assuming these are the images after correction with the Flats?

LP will generally show up as a gradient being brighter on one side (or top or bottom) than the other.

What you are seeing is vignetting which produces an image that is brighter in the center and darker towards the edges. Of course there is probably some LP mixed in too.

If the Flats are correcting the vignetting properly, then, after calibrating the images you will be left with only the LP gradient. In your case there is still a strong amount of the vignetting present (over or under corrected) even after correction with the flats. I'm having some problems with this too, but, mostly with the Red filter!! Not sure why one filter would have more problems than another. Still learning about this with my new ASI1600MM setup. Always something.

Steve

Totally true... There is always something!!

I think I'm going to play with the data a little bit, because I have one set of frames with 20k mean ADU values, but the other sets have 32k, and this is a mix. I think the 32k was overcorrecting, so I'm going to try to stack just the 20k frames and see what happens... I will share my findings here!

[Baskevo]

Okay so I ran a few tests... Tell me if you guys see a noticeable difference:

Test 1: Blue filter
Here is the original blue master, with both 32k and 20k flats applied to different nights, fully calibrated and pre-processed:

Here is the Blue filter with only the 20k flats applied from one night only:

Here is the blue filter with only the 32k flats applied from one night only:

Test 2: Green Filter
Original Green master, with both 32k and 20k flats applied to different nights, fully calibrated and pre-processed:

Here is the Green filter with only the 20k flats applied from one night only:

Here is the Green filter with only the 32k flats applied from one night only:

My thoughts:
It appears that the 20k flats have a lot less of the bright vignetting than the 32k flats, but on the Green filter it appears that there are still some irregularities in the 20k stacks (that spot in the bottom left corner), even though it is less pronounced. That spot appears in my light subs, but not in my flat frames... I'm not sure what I could do about that, because I would think that you would INCREASE the brightness of the flats to bring out that spot so it is removed in calibration...

Maybe it would be beneficial to go even lower with the flats, maybe around 15k...

What do you guys think? Do you all see what I'm seeing, or have I been looking at this for too long? lol

[UlteriorModem]

When you say 32K flats, are you applying 32 thousand flats?

[Baskevo]

When you say 32K flats, are you applying 32 thousand flats?

lol no it's the mean ADU value

[STEVE333]

Okay so I ran a few tests... Tell me if you guys see a noticeable difference:

Test 1: Blue filter
Here is the original blue master, with both 32k and 20k flats applied to different nights, fully calibrated and pre-processed:
Screen Shot 2020-03-24 at 4.51.41 PM.png

Here is the Blue filter with only the 20k flats applied from one night only:
Screen Shot 2020-03-24 at 4.20.36 PM.png

Here is the blue filter with only the 32k flats applied from one night only:
Screen Shot 2020-03-24 at 4.20.28 PM.png

Test 2: Green Filter
Original Green master, with both 32k and 20k flats applied to different nights, fully calibrated and pre-processed:
Screen Shot 2020-03-24 at 4.55.32 PM.png

Here is the Green filter with only the 20k flats applied from one night only:
G20k.png

Here is the Green filter with only the 32k flats applied from one night only:
G32k.png

My thoughts:
It appears that the 20k flats have a lot less of the bright vignetting than the 32k flats, but on the Green filter it appears that there are still some irregularities in the 20k stacks (that spot in the bottom left corner), even though it is less pronounced. That spot appears in my light subs, but not in my flat frames... I'm not sure what I could do about that, because I would think that you would INCREASE the brightness of the flats to bring out that spot so it is removed in calibration...

Maybe it would be beneficial to go even lower with the flats, maybe around 15k...

What do you guys think? Do you all see what I'm seeing, or have I been looking at this for too long? lol

That is a great test James! The fact that the 32K and 20K Flats produce different results is a real surprise to me. If the sensor/ADconverter combination was truly linear the two flats would have produced nearly identical results. The fact that the 20K flats produce the better results (at least to my eyes) implies that there may be an optimum ADU level for the flats. Whether it is higher or lower than 20K I don't know. I have seen several individuals using 12K flats for this sensor.

As a result of your tests i'm going to be doing some testing of my own if the clouds ever leave. Hopefully both of us can find the ADU level that produces good Flats corrections.

Thanks for sharing your efforts.

Steve

[Baskevo]

Okay so I ran a few tests... Tell me if you guys see a noticeable difference:

Test 1: Blue filter
Here is the original blue master, with both 32k and 20k flats applied to different nights, fully calibrated and pre-processed:
Screen Shot 2020-03-24 at 4.51.41 PM.png

Here is the Blue filter with only the 20k flats applied from one night only:
Screen Shot 2020-03-24 at 4.20.36 PM.png

Here is the blue filter with only the 32k flats applied from one night only:
Screen Shot 2020-03-24 at 4.20.28 PM.png

Test 2: Green Filter
Original Green master, with both 32k and 20k flats applied to different nights, fully calibrated and pre-processed:
Screen Shot 2020-03-24 at 4.55.32 PM.png

Here is the Green filter with only the 20k flats applied from one night only:
G20k.png

Here is the Green filter with only the 32k flats applied from one night only:
G32k.png

My thoughts:
It appears that the 20k flats have a lot less of the bright vignetting than the 32k flats, but on the Green filter it appears that there are still some irregularities in the 20k stacks (that spot in the bottom left corner), even though it is less pronounced. That spot appears in my light subs, but not in my flat frames... I'm not sure what I could do about that, because I would think that you would INCREASE the brightness of the flats to bring out that spot so it is removed in calibration...

Maybe it would be beneficial to go even lower with the flats, maybe around 15k...

What do you guys think? Do you all see what I'm seeing, or have I been looking at this for too long? lol

That is a great test James! The fact that the 32K and 20K Flats produce different results is a real surprise to me. If the sensor/ADconverter combination was truly linear the two flats would have produced nearly identical results. The fact that the 20K flats produce the better results (at least to my eyes) implies that there may be an optimum ADU level for the flats. Whether it is higher or lower than 20K I don't know. I have seen several individuals using 12K flats for this sensor.

As a result of your tests i'm going to be doing some testing of my own if the clouds ever leave. Hopefully both of us can find the ADU level that produces good Flats corrections.

Thanks for sharing your efforts.

Steve

Steve,

I'm glad I'm not the only one who sees the difference! and that I can help fellow APers! Please keep me posted on your testing, if you get around to it! I think I'm going to continue with 20k for this next object, and see if it makes a difference, then I will try 15k, and see how that goes!

What is the most difficult part is if knowing whether or not you stacks could be better if you had better calibration... For example, my narrowband filters are clean, so I don't get a lot of dust motes to tell me whether I have calibrated correctly or not. I have to rely on the vignetting, which even then I don't see a lot of. But my Oiii frames are still very hazy and has a lot of noise and gradients, and it is hard to tell if it can get fixed from better flats.

[UlteriorModem]

I am also surprised that flats of different adu values would have different results. I believed that the adu number of the individual flats was not really of great siginifigance. As long as they showed a good 'profile' of your optical train and were not saturated they should still be good!?

Baskeyo, could you describe to us the process you are using to calibrate? Are you using darks and bias as well? Are you scaling the darks?

I typically use Master flats that are already calibrated.

[Baskevo]

I am also surprised that flats of different adu values would have different results. I believed that the adu number of the individual flats was not really of great siginifigance. As long as they showed a good 'profile' of your optical train and were not saturated they should still be good!?

Baskeyo, could you describe to us the process you are using to calibrate? Are you using darks and bias as well? Are you scaling the darks?

I typically use Master flats that are already calibrated.

I spent about a month last fall reading and learning everything I possibly could about flats... From my understanding, the ADU value is a better indication of how good your flats are than the histogram is, and the profile is not the only thing that matters. I thought I was taking good flats for a while but they just would not help with my DSLR, until I started using ADU values to measure my flats, then they were perfect.
They have worked for me for the most part, just not on LRGB filters, at least not at the higher mean ADU value. I do not use bias, because they do not work well with the ASI1600mm, so I use flats, darks, and dark flats. I run the Weighted BPP script in PI twice, once to calibrate only, I run through subframe selector to remove bad frames, then I run WBPP script again without calibration to stack. It has worked well so far, and I get similar results when I experiment with DSS as well.

[UlteriorModem]

I see.

Might try calibrating the flats outside of the scripts with the calibration routine, then stacking manually creating a 'Master Flat" and see if you get any different results.

If you have Kellner's book he describes the manual calibration and integration of calibration files in great detail.

One thing about calibration of files outside of the script you can set things like "Weight=0 Dont care" etc.

[STEVE333]

I'm glad I'm not the only one who sees the difference! and that I can help fellow APers! Please keep me posted on your testing, if you get around to it! I think I'm going to continue with 20k for this next object, and see if it makes a difference, then I will try 15k, and see how that goes!

What is the most difficult part is if knowing whether or not you stacks could be better if you had better calibration... For example, my narrowband filters are clean, so I don't get a lot of dust motes to tell me whether I have calibrated correctly or not. I have to rely on the vignetting, which even then I don't see a lot of. But my Oiii frames are still very hazy and has a lot of noise and gradients, and it is hard to tell if it can get fixed from better flats.

I agree - how well are the Flats working??

I realized there was a problem with the flats when the corners of my image turned "greenish" when stretched. Clearly the green was over-corrected, or, the red was under-corrected. The Green and Blue "corrected" images looked to be uniform from center to edged, but, the Red image appeared to be dimmer in the corners. That is what makes me think the Red channel is the culprit.

To make matters worse, I'm just in the learning phase of determining the proper exposure time for each filter. I'm hoping that the graph I posted recently showing the proper mean ADU target will help to standardize my exposures. Then, hopefully, I can find the proper ADU target for the Flats. My God this is a real learning process!! I never realized how much work it would be to switch from the DSLR to NB. I hope there is light at the end of the tunnel. Right now, I'm not sure I see the tunnel.

Steve

[Baskevo]

I think I might have discovered my issue! The darks I was using were taken at 139 gain, but my frames were shot at 0 gain. Could that cause this issue? Flats and Dark flats still matched...

[UlteriorModem]

I dont know for sure if that would be your issue. But they should match in gain, offset, etc.

Why 0 gain? That may be more of the issue than anything else. Unity gain is 139 with an offset of 21.

[Baskevo]

I dont know for sure if that would be your issue. But they should match in gain, offset, etc.

Why 0 gain? That may be more of the issue than anything else. Unity gain is 139 with an offset of 21.

I was experimenting with different gain values as recommended by other AP'ers. I shoot at lower gain with LRGB filters because of my bortle 8. That should not be causing the issue at all.

https://www.cloudynights.com/topic/5738 ... be-qhy163/

[UlteriorModem]

Lower gain and filters?

Filters already attenuate the signal, and they say to use lower gain?

Yes I have read that thread and frankly I used to putz around with gain and offset settings.

But finally decided to just set it at unity and go with that. Besides it makes (as you have learned) the calibration files just another level of difficulty.

But that is just my opinion.