TheSkySearchers

Posted: **Tue Sep 08, 2020 10:04 pm**

Hopefully someone here can help me understand the issue that I am having.

I recently started using dark flats for calibration files instead of bias frames. I use a Nikon D5300 dlsr (cmos).

The last image that I processed (Eastern Veil) was the first time that I tried dark flats and I had no problems.

Today, I tried dark flats for the second time with the Western Veil Nebula (dark flats exposure for 1 second at iso 400 just like the flats). Same iso as my light frames.

This is the stacked image that I see in deepskystacker. Looks abnormally bright (usually the stacked image is very dark with just stars showing).

When I open the DSS FTS file in Startools and run Autodev (initial stretch), this is what I see.

Completely abnormal looking stacked image and stretch in Startools. Extreme vignetting (or reverse vignetting).

If I replace the dark flats in deepskystacker with regular bias frames (1/4000 sec at iso 400), my stacked image in dss looks normal (dark).

This is the initial stretch that I see in Startools (Aurodev). This looks normal.

It took me awhile to finally arrive at the fact that the dark flats were causing my issue since I successfully used them my last processing.

NOTE: The same light frames, master dark, and flats were used in both stacks. The only difference was the dark flats and bias.

NOTE:

This is what one of the dark flats look like.

Took lots of stacks to troubleshoot this today, but as it turns out, the data is poor anyway. Last night I measured the sky quality at 18.05 (Bortle 8) which was unusually poor for my backyard. Had me really going with the dark flats though!

I have a ASI 533 mc pro on the way with an L Enhance to follow. Hopefully, this will help out!

Thank you for any help that you can give me.

Posted: **Tue Sep 08, 2020 10:57 pm**

Hi Jim - Maybe I can help.

I also have a CMOS sensor but mine is cooled. However, cooled or not, I believe the calibration approach is the same.

1) Capture Flats and Dark Flats with the exact same exposure time (you said you did this).
1a) Combine the Dark Flats to create a Master Dark Flat.
1b) Calibrate flats by subtracting the Master Dark Flat. Don't use any Bias frames.
2) Make Master Dark frame by just averaging the individual dark frames. Don't use any Bias frames!
3) Calibrate Light frames using only Master Flat and Master Dark. Don't use any Bias frames!

I don't know if this will work for your DSLR, but, it works well for my CMOS sensor.

Hope this helps.

Steve

Posted: **Tue Sep 08, 2020 11:22 pm**

Hi Steve and thanks for the response!

In your step 3, you mention calibrate the light frames with the master flat and the master dark. No bias. What about the dark flats?

If the dark flats aren't used in the calibration, how will the bias signal be calibrated?

Thanks!

Posted: **Tue Sep 08, 2020 11:31 pm**

Here are graphics depicting all of the DSS operations. Any or all of these should work.

Cheers,
JT

PS, These procedures are now in the correct order.

Posted: **Wed Sep 09, 2020 12:06 am**

JayTee wrote: ↑Tue Sep 08, 2020 11:31 pm Here are graphics depicting all of the DSS operations.

Cheers,
JT

PS, sorry these kind of got shown out of order. Consider the bottom graphic first.

Thanks a bunch JT!

So, It looks like I need all four. Lights, darks, flats, and dark flats. I thought that I had read somewhere that the dark flats take the place of the bias since they already contain the bias signal. I will run all four through dss.

I appreciate the help and understanding!

Posted: **Wed Sep 09, 2020 12:40 am**

Thanks for the graphics Jim.

The second graphic is what you want. The one with NO Bias frames.

Good luck.

Steve

Posted: **Wed Sep 09, 2020 1:08 am**

STEVE333 wrote: ↑Wed Sep 09, 2020 12:40 am Thanks for the graphics Jim.

The second graphic is what you want. The one with NO Bias frames.

Good luck.

Steve

Thanks Steve. Unfortunately, that is the combination that I used originally that produced the unusual stack.

I’m sure that it is something that I did wrong, but I went over it all and can’t find anything wrong.

The only way thatI can get a good stacked fts file is to include the bias. Really doesn’t make much difference with this image since it is way too poor to pursue. Just trying to get it right for 5he next session.

Thanks a lot for your help!

Posted: **Wed Sep 09, 2020 3:23 am**

Jim,

Any of those three procedures are viable processes for your image stack.

Also, did you correctly label the images in DSS? In other words, you told DSS that the Dark Flats were actually Dark Flats.

Cheers,
JT

Posted: **Wed Sep 09, 2020 3:30 am**

Juno16 wrote: ↑Wed Sep 09, 2020 1:08 am
STEVE333 wrote: ↑Wed Sep 09, 2020 12:40 am Thanks for the graphics Jim.

The second graphic is what you want. The one with NO Bias frames.

Good luck.

Steve

Thanks Steve. Unfortunately, that is the combination that I used originally that produced the unusual stack.

I’m sure that it is something that I did wrong, but I went over it all and can’t find anything wrong.

The only way thatI can get a good stacked fts file is to include the bias. Really doesn’t make much difference with this image since it is way too poor to pursue. Just trying to get it right for 5he next session.

Thanks a lot for your help!

When I was using a DSLR (Canon T3) I always used Bias frames when doing calibration and never used Dark Flats. I got reasonable results for a few years. However, when I got my new cooled CMOS camera (ASI1600MM) and tried to calibrate it the same way I had been calibrating my DSLR (with Bias frames) I had terrible problems. That's when I learned about Dark Flats.

The reason for mentioning all this is that maybe DSLR CMOS chips/electronics don't behave the same way as the new CMOS chips/electronics used in the cooled cameras. Wish I knew the answer.

Good luck in solving your issues. Wish I could have been of some help.

Steve

Posted: **Wed Sep 09, 2020 11:42 am**

STEVE333 wrote: ↑Wed Sep 09, 2020 3:30 am
Juno16 wrote: ↑Wed Sep 09, 2020 1:08 am
STEVE333 wrote: ↑Wed Sep 09, 2020 12:40 am Thanks for the graphics Jim.

The second graphic is what you want. The one with NO Bias frames.

Good luck.

Steve

Thanks Steve. Unfortunately, that is the combination that I used originally that produced the unusual stack.

I’m sure that it is something that I did wrong, but I went over it all and can’t find anything wrong.

The only way thatI can get a good stacked fts file is to include the bias. Really doesn’t make much difference with this image since it is way too poor to pursue. Just trying to get it right for 5he next session.

Thanks a lot for your help!

When I was using a DSLR (Canon T3) I always used Bias frames when doing calibration and never used Dark Flats. I got reasonable results for a few years. However, when I got my new cooled CMOS camera (ASI1600MM) and tried to calibrate it the same way I had been calibrating my DSLR (with Bias frames) I had terrible problems. That's when I learned about Dark Flats.

The reason for mentioning all this is that maybe DSLR CMOS chips/electronics don't behave the same way as the new CMOS chips/electronics used in the cooled cameras. Wish I knew the answer.

Good luck in solving your issues. Wish I could have been of some help.

Steve

Steve, You have offered me lots of very helpful feedback now and in the past and I appreciate it!

I also have always used bias frames and not dark flats with my dslr up until my last image of the East Veil. I used dark flats and all went well.

I have been trying many different things to get to the root of this issue and just this morning, I think that I have found the problem.

When I processed the first stack and ran into the weird light stacked image along with the strange vignetted looking stretch in Startools, I grabbed a master dark from a previous session that was of similar temperature.

I re-stacked this morning using the actual dark frames (not the master dark) from the East Veil data set. The result was a normal very dark stacked image that stretched normally in Startools. I replaced the single dark frames with the master dark and re-stacked. The results were again poor.
Usually, I never use the master darks since I just make one stack of the original single frames and go with that. Luckily, I still save everything (I have 6 TB storage on my desktop). Very weird, but at least a cause was identified.

I feel much better knowing the cause even though it really doesn't make much sense. However, it is definately the cause.

Unfortunately, the data is still quite noisy. The sky quality was very poor that evening and has been getting progressively worse since the area is "lighting up" more than it used to be. I have big hopes that the added sensitivity of the new 533 along with the selective bands of the L-enhance will help some. Might even have to get an Ha filter sometime in the future.

Looks like the dark flat replacement of the bias frames does indeed work with dslr cmos sensors/electronics.

Again, thanks for the help Steve!

Posted: **Wed Sep 09, 2020 11:50 am**

6TB!!!!

Nice.

Regards

Graeme

Posted: **Wed Sep 09, 2020 12:34 pm**

JayTee wrote: ↑Wed Sep 09, 2020 3:23 am Jim,

Any of those three procedures are viable processes for your image stack.

Also, did you correctly label the images in DSS? In other words, you told DSS that the Dark Flats were actually Dark Flats.

Cheers,
JT

Yes, wnen I saw the issue, I did verify with two iterations in dss.

Like I told Steve, I confirmed that there is an issue with the master dark. When I reprocesses in dss using the individual dark frames as opposed to the master dark, the issue went away. I did both 2x just to be sure.
Strange, but at least I know the source of the issue even if it really doesn’t make sense,.

Thanks a bunch JT for your helpful responses!

Posted: **Wed Sep 09, 2020 1:28 pm**

Interesting find, Jim. I tend to reuse master darks created in DSS if temperature matches closely. I have had issues in the past with darks where I have found that stacking without any dark or master dark can give better results than using the master dark, but hadn't made the connection. Glad you got it sorted out! I'll have to pay attention to the possible issue going forward.

For my last DSLR session (IC1396) I used darks flats instead of bias and was happy with the result. My thought is that all calibration frames contain the bias signal, so subtracting darks from lights and dark flats from flats subtracts out the bias contribution without actually having to include the bias frames in the stack. To my mind that just eliminates one more potential source of random noise in the process.

Bob

Posted: **Wed Sep 09, 2020 3:56 pm**

Juno16 wrote: ↑Wed Sep 09, 2020 11:42 am
I re-stacked this morning using the actual dark frames (not the master dark) from the East Veil data set. The result was a normal very dark stacked image that stretched normally in Startools. I replaced the single dark frames with the master dark and re-stacked. The results were again poor.
Usually, I never use the master darks since I just make one stack of the original single frames and go with that. Luckily, I still save everything (I have 6 TB storage on my desktop). Very weird, but at least a cause was identified.

I feel much better knowing the cause even though it really doesn't make much sense. However, it is definately the cause.

Unfortunately, the data is still quite noisy. The sky quality was very poor that evening and has been getting progressively worse since the area is "lighting up" more than it used to be. I have big hopes that the added sensitivity of the new 533 along with the selective bands of the L-enhance will help some. Might even have to get an Ha filter sometime in the future.

Looks like the dark flat replacement of the bias frames does indeed work with dslr cmos sensors/electronics.

Again, thanks for the help Steve!

You're welcome Jim. Thanks for posting the results of your testing to find the problem. It is comforting to know that CMOS behaves like CMOS whether it is a new cooled sensor or a DSLR.

That was a good catch to determine the Master Dark was the cause of the problem. Very strange. With my new cooled sensor I have been using the same MasterDark frame for months with no problems. Good thing, because, with my 10 min exposures it takes quite a bit of time to take a bunch of Dark Frames!

Look forward to seeing "First LIght" with your 533. By the way, I'm assuming the 533 is a CMOS sensor, so, now you know the calibration procedure!

Steve

Posted: **Wed Sep 09, 2020 10:37 pm**

bobharmony wrote: ↑Wed Sep 09, 2020 1:28 pm Interesting find, Jim. I tend to reuse master darks created in DSS if temperature matches closely. I have had issues in the past with darks where I have found that stacking without any dark or master dark can give better results than using the master dark, but hadn't made the connection. Glad you got it sorted out! I'll have to pay attention to the possible issue going forward.

For my last DSLR session (IC1396) I used darks flats instead of bias and was happy with the result. My thought is that all calibration frames contain the bias signal, so subtracting darks from lights and dark flats from flats subtracts out the bias contribution without actually having to include the bias frames in the stack. To my mind that just eliminates one more potential source of random noise in the process.

Bob

It is kind of weird Bob. I ran 12 stacks trying different things and kept a log because it got beyond "simple". I also reuse master darks. This one was from the NGC 7000 session and I only ran one stack (where I used the raw dark frames).

I confirmed 2x each stacking with individual darks and stacking with the master and the issue repeated. Strange thing for sure, but confirmed.

Dark flats take a small added amount of work from shooting bias, but that will be my future path.

STEVE333 wrote: ↑Wed Sep 09, 2020 3:56 pm

You're welcome Jim. Thanks for posting the results of your testing to find the problem. It is comforting to know that CMOS behaves like CMOS whether it is a new cooled sensor or a DSLR.

That was a good catch to determine the Master Dark was the cause of the problem. Very strange. With my new cooled sensor I have been using the same MasterDark frame for months with no problems. Good thing, because, with my 10 min exposures it takes quite a bit of time to take a bunch of Dark Frames!

Look forward to seeing "First LIght" with your 533. By the way, I'm assuming the 533 is a CMOS sensor, so, now you know the calibration procedure!

Steve

For sure, me too! The lp has me worried some since the last few sessions have been rough. Hoping that the new camera will help with increased sensitivity and Agena Astro told me that the L-enhance should be in at the end of the week. Duoband should help too.

Thanks again for the help Steve.

Posted: **Tue Sep 22, 2020 12:05 am**

Just as a tiny point, it's actually flat-darks, not dark-flats. You're taking a dark frame matching your flat, so it's a flat-dark.

Posted: **Tue Sep 22, 2020 12:07 am**

And the other thing, your flats do not need to be at the same ISO as your lights (or darks).

Posted: **Tue Sep 22, 2020 12:17 am**

Thanks Stuart on both points.

The only reason that I called them dark flats is because that is what they are called in DeepSkyStacker.

Posted: **Tue Sep 22, 2020 1:10 am**

FWIW, here is the verbiage from the DSS FAQ regarding all the different frames.

Light Frames
The Light Frames are the images that contain the real information: images of galaxies, nebula...
This is what you want to stack.

Dark Frames and Dark Flat Frames
The Dark Frames are used to remove the dark signal from the light frames (or the flat frames for the Dark Flat frames).
With DSLRs and CCD Cameras, the CMOS or CCD sensor is generating a dark signal depending on the exposure time, temperature, and ISO speed (DSLR only).
To remove the dark signal from the light frames you use a dark frame that contains only the dark signal.

The best way to create the dark frames is to shoot pictures in the dark (hence the name) by covering the lens.
The dark frames must be created with the exposure time, temperature, and ISO speed of the light frames (resp. flat frames).
Since the temperature is important try to shoot dark frames at the end or during your imaging session.

Take a few of them (between 10 and 20 is usually enough). DeepSkyStacker will combine them automatically to create and use a clean master dark or master dark flat.

Bias Frames (aka Offset Frames)
The Bias/Offset Frames are used to remove the CCD or CMOS chip readout signal from the light frames.
Each CCD or CMOS chip is generating a readout signal which is a signal created by the electronic just by reading the content of the chip.

It's very easy to create bias/offset frames: just take the shortest possible exposure (it may be 1/4000s or 1/8000s depending on your camera) in the dark by covering the lens.
The bias frames must be created with the ISO speed of the light frames. The temperature is not important.

Take a few of them (between 10 and 20 is usually enough). DeepSkyStacker will combine them automatically to create and use a clean master bias/offset frame

Flat Frames
The Flat Frames are used to correct the vignetting and uneven field illumination created by dust or smudges in your optical train.

To create good flat frames it is very important to not remove your camera from your telescope before taking them (including not changing the focus).
You can use a lot of different methods (including using a flats box) but I found that the simplest way is to put a white T-shirt in front of your telescope and smooth out the folds. Then shoot something luminous (a flash, a bright white light, the sky at dawn...) and let the camera decide of the exposure time (Av mode),

The flat frames should be created with the ISO speed of the light frames. The temperature is not important.

Take a few of them (between 10 and 20 is usually enough). DeepSkyStacker will combine them automatically to create and use a clean master flat frame.

Cheers,
JT

Posted: **Tue Sep 22, 2020 1:40 am**

Stuart wrote: ↑Tue Sep 22, 2020 12:07 am And the other thing, your flats do not need to be at the same ISO as your lights (or darks).

DSS is written by a non-native English speaker, and he got it mixed up. It's caused a lot of confusion.

TheSkySearchers

Dark Flats Issue

Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue

Re: Dark Flats Issue