Disk Space Requirements For Pixinsight

Discuss how you are able to get those fantastic images!!!
Post Reply
User avatar
Juno16 United States of America
Universal Ambassador
Articles: 0
Offline
Posts: 8195
Joined: Sun May 12, 2019 3:13 pm
4
Location: Mississippi Gulf Coast
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

Disk Space Requirements For Pixinsight

#1

Post by Juno16 »


Hi PI Users,

I upgraded to a commercial PI license late last year (12/28/21). Since I have begun using PI, my disk space has been dwindling. I started with a 500g nvme m.2, 500g sata ssd and a 3TB hdd and I am now adding a 4TB hdd.

I know that PI generates a lot of data, but is it really all necessary if I use WBPP (like the registered folder) for all of my pre-processing?

What files do you normally delete and what files do you keep?

Thanks a bunch!
Jim

Scopes: Explore Scientific ED102 APO, Sharpstar 61 EDPH II APO, Samyang 135 F2 (still on the Nikon).
Mount: Skywatcher HEQ5 Pro with Rowan Belt Mod
Stuff: ASI EAF Focus Motor (x2), Orion 50mm Guide Scope, ZWO 30 mm Guide Scope, ASI 220mm min, ASI 120mm mini, Stellarview 0.8 FR/FF, Sharpstar 0.8 FR/FF, Mele Overloock 3C.
Camera/Filters/Software: ASI 533 mc pro, ASI 120mm mini, Orion SSAG, IDAS LPS D-1, Optolong L-Enhance, ZWO UV/IR Cut, N.I.N.A., Green Swamp Server, PHD2, Adobe Photoshop CC, Pixinsight.
Dog and best bud: Jack
Sky: Bortle 6-7
My Astrobin: https://www.astrobin.com/users/Juno16/
User avatar
STEVE333 United States of America
Inter-Galactic Ambassador
Articles: 0
Offline
Posts: 3466
Joined: Sat May 11, 2019 5:01 pm
4
Location: Santa Cruz, Ca, USA
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

Re: Disk Space Requirements For Pixinsight

#2

Post by STEVE333 »


Hi Jim - After WBPP I just retain the Masters and the Calibrated. If I capture more data later I can Integrate the new Calibrated with the old calibrated. So, no need to keep the Registered.

I'm starting to wonder if I need to keep my original Lights once I have run WBPP. To use them again I would need to have the original Darks and Flats which I don't always keep.

Anyway, that's my approach.

Steve
Steve King: Light Pollution (Bortle 5)
Telescope + Mount + Guiding: W.O. Star71-ii + iOptron CEM40 EC + Orion Magnificent Mini AutoGuider
Camera: ASI 1600MM Pro + EFW Filter Wheel + Chroma 3nm Siii, Ha, Oiii + ZWO LRGB Filters
Software: PHD2; APT; PixInsight ***** My AP website: www.steveking.pictures
Image
Image
User avatar
Juno16 United States of America
Universal Ambassador
Articles: 0
Offline
Posts: 8195
Joined: Sun May 12, 2019 3:13 pm
4
Location: Mississippi Gulf Coast
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

Re: Disk Space Requirements For Pixinsight

#3

Post by Juno16 »


STEVE333 wrote: Mon May 17, 2021 8:39 pm Hi Jim - After WBPP I just retain the Masters and the Calibrated. If I capture more data later I can Integrate the new Calibrated with the old calibrated. So, no need to keep the Registered.

I'm starting to wonder if I need to keep my original Lights once I have run WBPP. To use them again I would need to have the original Darks and Flats which I don't always keep.

Anyway, that's my approach.

Steve
Much appreciated Steve. Well, actually I had a 4TB hdd installed instead of the 3 that I thought. Now I’m up to 9 TB total.

Are the calibrated lights also registered?

I guess if you have the masters, you don’t need the darks, flats, or flat darks.

Thanks a lot Steve. That’s very helpful
Jim

Scopes: Explore Scientific ED102 APO, Sharpstar 61 EDPH II APO, Samyang 135 F2 (still on the Nikon).
Mount: Skywatcher HEQ5 Pro with Rowan Belt Mod
Stuff: ASI EAF Focus Motor (x2), Orion 50mm Guide Scope, ZWO 30 mm Guide Scope, ASI 220mm min, ASI 120mm mini, Stellarview 0.8 FR/FF, Sharpstar 0.8 FR/FF, Mele Overloock 3C.
Camera/Filters/Software: ASI 533 mc pro, ASI 120mm mini, Orion SSAG, IDAS LPS D-1, Optolong L-Enhance, ZWO UV/IR Cut, N.I.N.A., Green Swamp Server, PHD2, Adobe Photoshop CC, Pixinsight.
Dog and best bud: Jack
Sky: Bortle 6-7
My Astrobin: https://www.astrobin.com/users/Juno16/
User avatar
STEVE333 United States of America
Inter-Galactic Ambassador
Articles: 0
Offline
Posts: 3466
Joined: Sat May 11, 2019 5:01 pm
4
Location: Santa Cruz, Ca, USA
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

Re: Disk Space Requirements For Pixinsight

#4

Post by STEVE333 »


Juno16 wrote: Mon May 17, 2021 9:24 pm
STEVE333 wrote: Mon May 17, 2021 8:39 pm Hi Jim - After WBPP I just retain the Masters and the Calibrated. If I capture more data later I can Integrate the new Calibrated with the old calibrated. So, no need to keep the Registered.

I'm starting to wonder if I need to keep my original Lights once I have run WBPP. To use them again I would need to have the original Darks and Flats which I don't always keep.

Anyway, that's my approach.

Steve
Much appreciated Steve. Well, actually I had a 4TB hdd installed instead of the 3 that I thought. Now I’m up to 9 TB total.

Are the calibrated lights also registered?

I guess if you have the masters, you don’t need the darks, flats, or flat darks.

Thanks a lot Steve. That’s very helpful

You're welcome Jim.
The calibrated Lights are NOT registered.
Correct, if you have the masters then you don't need the darks or flats (or flat darks).

Have fun!

Steve
Steve King: Light Pollution (Bortle 5)
Telescope + Mount + Guiding: W.O. Star71-ii + iOptron CEM40 EC + Orion Magnificent Mini AutoGuider
Camera: ASI 1600MM Pro + EFW Filter Wheel + Chroma 3nm Siii, Ha, Oiii + ZWO LRGB Filters
Software: PHD2; APT; PixInsight ***** My AP website: www.steveking.pictures
Image
Image
User avatar
Graeme1858 Great Britain
Co-Administrator
Co-Administrator
Articles: 1
Offline
Posts: 7217
Joined: Mon Jun 24, 2019 7:16 pm
4
Location: North Kent, UK
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

I Broke The Forum.

Re: Disk Space Requirements For Pixinsight

#5

Post by Graeme1858 »


I'm not a PI user (yet!) But I have to keep an eye on the processing pc hard drive. The imaging pc keeps all the original captured files. OneDrive keeps a copy. Once transferred and processed, only the stacked fits and the calibration masters get kept. If I'm feeling ruthless or unhappy with a capture or the hard drive is getting full, I only keep the exported png file.

So, is PI disk space hungry?

Regards

Graeme
______________________________________________
Celestron 9.25 f10 SCT, f6.3FR, CGX mount.
ASI1600MM Pro, ASI294MC Pro, ASI224MC
ZWO EFW, ZWO OAG, ASI220MM Mini.
APM 11x70 ED APO Binoculars.

https://www.averywayobservatory.co.uk/
User avatar
Juno16 United States of America
Universal Ambassador
Articles: 0
Offline
Posts: 8195
Joined: Sun May 12, 2019 3:13 pm
4
Location: Mississippi Gulf Coast
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

Re: Disk Space Requirements For Pixinsight

#6

Post by Juno16 »


STEVE333 wrote: Tue May 18, 2021 3:42 am
You're welcome Jim.
The calibrated Lights are NOT registered.
Correct, if you have the masters then you don't need the darks or flats (or flat darks).

Have fun!

Steve
Thanks a lot Steve. I will keep the lights and masters and delete the rest. If I revisit the data again, I will just run WBPP.
Jim

Scopes: Explore Scientific ED102 APO, Sharpstar 61 EDPH II APO, Samyang 135 F2 (still on the Nikon).
Mount: Skywatcher HEQ5 Pro with Rowan Belt Mod
Stuff: ASI EAF Focus Motor (x2), Orion 50mm Guide Scope, ZWO 30 mm Guide Scope, ASI 220mm min, ASI 120mm mini, Stellarview 0.8 FR/FF, Sharpstar 0.8 FR/FF, Mele Overloock 3C.
Camera/Filters/Software: ASI 533 mc pro, ASI 120mm mini, Orion SSAG, IDAS LPS D-1, Optolong L-Enhance, ZWO UV/IR Cut, N.I.N.A., Green Swamp Server, PHD2, Adobe Photoshop CC, Pixinsight.
Dog and best bud: Jack
Sky: Bortle 6-7
My Astrobin: https://www.astrobin.com/users/Juno16/
User avatar
Juno16 United States of America
Universal Ambassador
Articles: 0
Offline
Posts: 8195
Joined: Sun May 12, 2019 3:13 pm
4
Location: Mississippi Gulf Coast
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

Re: Disk Space Requirements For Pixinsight

#7

Post by Juno16 »


Graeme1858 wrote: Tue May 18, 2021 4:59 am I'm not a PI user (yet!) But I have to keep an eye on the processing pc hard drive. The imaging pc keeps all the original captured files. OneDrive keeps a copy. Once transferred and processed, only the stacked fits and the calibration masters get kept. If I'm feeling ruthless or unhappy with a capture or the hard drive is getting full, I only keep the exported png file.

So, is PI disk space hungry?

Regards

Graeme
Thanks Graeme. At least for me PI is disk space hungry. For me, a folder for one target can be 4G, but I’m still learning, I save copies along the way. PI operates in their .xisf file format and typically the file size is just over 100MB per file.
I should be good for awhile with 9TB. Disk space (hdd) is cheap though. The last 4TB hdd was only $100.
Jim

Scopes: Explore Scientific ED102 APO, Sharpstar 61 EDPH II APO, Samyang 135 F2 (still on the Nikon).
Mount: Skywatcher HEQ5 Pro with Rowan Belt Mod
Stuff: ASI EAF Focus Motor (x2), Orion 50mm Guide Scope, ZWO 30 mm Guide Scope, ASI 220mm min, ASI 120mm mini, Stellarview 0.8 FR/FF, Sharpstar 0.8 FR/FF, Mele Overloock 3C.
Camera/Filters/Software: ASI 533 mc pro, ASI 120mm mini, Orion SSAG, IDAS LPS D-1, Optolong L-Enhance, ZWO UV/IR Cut, N.I.N.A., Green Swamp Server, PHD2, Adobe Photoshop CC, Pixinsight.
Dog and best bud: Jack
Sky: Bortle 6-7
My Astrobin: https://www.astrobin.com/users/Juno16/
User avatar
yobbo89 Australia
Moderator
Moderator
Articles: 0
Offline
Posts: 2561
Joined: Sat May 11, 2019 7:44 pm
4
Location: australia qld brisbane
Status:
Offline

TSS Photo of the Day

Re: Disk Space Requirements For Pixinsight

#8

Post by yobbo89 »


I deleate all xsif but the ones i want to rework on or are just too time consuming to recalibrate ie like a 9 panel and keep the original raw files output from sgpro or sharpcap., the file size range from 128mb for a single panel per filter and over 900mb for a multi panel per filter..., a big hd space consumption.. you should atleast try and keep most of your data, it is good learning curve and you can go back to an old data set and give it a repro with newer skills obtained down the road..
scopes :gso/bintel f4 12"truss tube, bresser messier ar127s /skywatcher 10'' dob,meade 12'' f10 lx200 sct
cameras : asi 1600mm-c/asi1600mm-c,asi120mc,prostar lp guidecam, nikkon d60, sony a7,asi 290 mm
mounts : eq6 pro/eq8/mesu 200 v2
filters : 2'' astronomik lp/badder lrgb h-a,sII,oIII,h-b,Baader Solar Continuum, chroma 3nm ha,sii,oiii,nii,rgb,lowglow,uv/ir,Thousand Oaks Solar Filter,1.25'' #47 violet,pro planet 742 ir,pro planet 807 ir,pro planet 642 bp ir.
extras : skywatcher f4 aplanatic cc, Baader MPCC MKIII Coma Corrector,Orion Field Flattener,zwo 1.25''adc.starlight maxi 2" 9x filter wheel,tele vue 2x barlow .

Image
User avatar
STEVE333 United States of America
Inter-Galactic Ambassador
Articles: 0
Offline
Posts: 3466
Joined: Sat May 11, 2019 5:01 pm
4
Location: Santa Cruz, Ca, USA
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

Re: Disk Space Requirements For Pixinsight

#9

Post by STEVE333 »


yobbo89 wrote: Tue May 18, 2021 12:43 pm I deleate all xsif but the ones i want to rework on or are just too time consuming to recalibrate ie like a 9 panel and keep the original raw files output from sgpro or sharpcap., the file size range from 128mb for a single panel per filter and over 900mb for a multi panel per filter..., a big hd space consumption.. you should atleast try and keep most of your data, it is good learning curve and you can go back to an old data set and give it a repro with newer skills obtained down the road..

One thing I've been thinking about is whether it is necessary to save the original Raw Lights. After running WBPP (or however I calibrates the Lights) I then have calibrated Lights and Masters (Master OSC image or Master NB images). All future processing can be performed with the Masters, or, the calibrated Lights can be combined with any new calibrated Lights to create new Masters. It doesn't seem that the original Raw Lights are needed anymore. Am I missing something?

Steve
Steve King: Light Pollution (Bortle 5)
Telescope + Mount + Guiding: W.O. Star71-ii + iOptron CEM40 EC + Orion Magnificent Mini AutoGuider
Camera: ASI 1600MM Pro + EFW Filter Wheel + Chroma 3nm Siii, Ha, Oiii + ZWO LRGB Filters
Software: PHD2; APT; PixInsight ***** My AP website: www.steveking.pictures
Image
Image
User avatar
Juno16 United States of America
Universal Ambassador
Articles: 0
Offline
Posts: 8195
Joined: Sun May 12, 2019 3:13 pm
4
Location: Mississippi Gulf Coast
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

Re: Disk Space Requirements For Pixinsight

#10

Post by Juno16 »


STEVE333 wrote: Tue May 18, 2021 5:26 pm
yobbo89 wrote: Tue May 18, 2021 12:43 pm I deleate all xsif but the ones i want to rework on or are just too time consuming to recalibrate ie like a 9 panel and keep the original raw files output from sgpro or sharpcap., the file size range from 128mb for a single panel per filter and over 900mb for a multi panel per filter..., a big hd space consumption.. you should atleast try and keep most of your data, it is good learning curve and you can go back to an old data set and give it a repro with newer skills obtained down the road..

One thing I've been thinking about is whether it is necessary to save the original Raw Lights. After running WBPP (or however I calibrates the Lights) I then have calibrated Lights and Masters (Master OSC image or Master NB images). All future processing can be performed with the Masters, or, the calibrated Lights can be combined with any new calibrated Lights to create new Masters. It doesn't seem that the original Raw Lights are needed anymore. Am I missing something?

Steve
Thanks Rob for your methodology @yobbo89

Now that you have enlightened me Steve and Rob, I believe that you have good approaches to keeping the space used for processed (actually all) image files neat and orderly.

Right now, I will begin cleaning up the 500G ssd that I have my image files on. Once cleaned up, I will transfer to one of the big hdd's. I will have plenty of room for future storage and disk image backups.

Thanks a lot!
Jim

Scopes: Explore Scientific ED102 APO, Sharpstar 61 EDPH II APO, Samyang 135 F2 (still on the Nikon).
Mount: Skywatcher HEQ5 Pro with Rowan Belt Mod
Stuff: ASI EAF Focus Motor (x2), Orion 50mm Guide Scope, ZWO 30 mm Guide Scope, ASI 220mm min, ASI 120mm mini, Stellarview 0.8 FR/FF, Sharpstar 0.8 FR/FF, Mele Overloock 3C.
Camera/Filters/Software: ASI 533 mc pro, ASI 120mm mini, Orion SSAG, IDAS LPS D-1, Optolong L-Enhance, ZWO UV/IR Cut, N.I.N.A., Green Swamp Server, PHD2, Adobe Photoshop CC, Pixinsight.
Dog and best bud: Jack
Sky: Bortle 6-7
My Astrobin: https://www.astrobin.com/users/Juno16/
User avatar
yobbo89 Australia
Moderator
Moderator
Articles: 0
Offline
Posts: 2561
Joined: Sat May 11, 2019 7:44 pm
4
Location: australia qld brisbane
Status:
Offline

TSS Photo of the Day

Re: Disk Space Requirements For Pixinsight

#11

Post by yobbo89 »


STEVE333 wrote: Tue May 18, 2021 5:26 pm
yobbo89 wrote: Tue May 18, 2021 12:43 pm I deleate all xsif but the ones i want to rework on or are just too time consuming to recalibrate ie like a 9 panel and keep the original raw files output from sgpro or sharpcap., the file size range from 128mb for a single panel per filter and over 900mb for a multi panel per filter..., a big hd space consumption.. you should atleast try and keep most of your data, it is good learning curve and you can go back to an old data set and give it a repro with newer skills obtained down the road..

One thing I've been thinking about is whether it is necessary to save the original Raw Lights. After running WBPP (or however I calibrates the Lights) I then have calibrated Lights and Masters (Master OSC image or Master NB images). All future processing can be performed with the Masters, or, the calibrated Lights can be combined with any new calibrated Lights to create new Masters. It doesn't seem that the original Raw Lights are needed anymore. Am I missing something?

Steve
I'd say it gives you more options, ie to replace with newer data in the stack and push out any average frames,it's one thing I'm trying to do with my 9 panel, a few panels had a bad sky background from clouds but some good frames where in the stack. I think there are other options to be used in the stacking method,drizzle , average,mean,sigma combination ect.. i don't think there is a work around for that if one had already stacked and deleted the raw files and saved the master, you could say the same for planetary imaging, stacks are a hit or miss,sometimes it can take a few trys to get a good result, there are a few variables to play with, stack size, reference frames, alignment box ectt. A full rework from raw files might yield a better final image? :Think:
scopes :gso/bintel f4 12"truss tube, bresser messier ar127s /skywatcher 10'' dob,meade 12'' f10 lx200 sct
cameras : asi 1600mm-c/asi1600mm-c,asi120mc,prostar lp guidecam, nikkon d60, sony a7,asi 290 mm
mounts : eq6 pro/eq8/mesu 200 v2
filters : 2'' astronomik lp/badder lrgb h-a,sII,oIII,h-b,Baader Solar Continuum, chroma 3nm ha,sii,oiii,nii,rgb,lowglow,uv/ir,Thousand Oaks Solar Filter,1.25'' #47 violet,pro planet 742 ir,pro planet 807 ir,pro planet 642 bp ir.
extras : skywatcher f4 aplanatic cc, Baader MPCC MKIII Coma Corrector,Orion Field Flattener,zwo 1.25''adc.starlight maxi 2" 9x filter wheel,tele vue 2x barlow .

Image
User avatar
STEVE333 United States of America
Inter-Galactic Ambassador
Articles: 0
Offline
Posts: 3466
Joined: Sat May 11, 2019 5:01 pm
4
Location: Santa Cruz, Ca, USA
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

Re: Disk Space Requirements For Pixinsight

#12

Post by STEVE333 »


yobbo89 wrote: Tue May 18, 2021 8:05 pm
STEVE333 wrote: Tue May 18, 2021 5:26 pm
yobbo89 wrote: Tue May 18, 2021 12:43 pm I deleate all xsif but the ones i want to rework on or are just too time consuming to recalibrate ie like a 9 panel and keep the original raw files output from sgpro or sharpcap., the file size range from 128mb for a single panel per filter and over 900mb for a multi panel per filter..., a big hd space consumption.. you should atleast try and keep most of your data, it is good learning curve and you can go back to an old data set and give it a repro with newer skills obtained down the road..

One thing I've been thinking about is whether it is necessary to save the original Raw Lights. After running WBPP (or however I calibrates the Lights) I then have calibrated Lights and Masters (Master OSC image or Master NB images). All future processing can be performed with the Masters, or, the calibrated Lights can be combined with any new calibrated Lights to create new Masters. It doesn't seem that the original Raw Lights are needed anymore. Am I missing something?

Steve
I'd say it gives you more options, ie to replace with newer data in the stack and push out any average frames,it's one thing I'm trying to do with my 9 panel, a few panels had a bad sky background from clouds but some good frames where in the stack. I think there are other options to be used in the stacking method,drizzle , average,mean,sigma combination ect.. i don't think there is a work around for that if one had already stacked and deleted the raw files and saved the master, you could say the same for planetary imaging, stacks are a hit or miss,sometimes it can take a few trys to get a good result, there are a few variables to play with, stack size, reference frames, alignment box ectt. A full rework from raw files might yield a better final image? :Think:

I get it. Thanks for the ideas.

Steve
Steve King: Light Pollution (Bortle 5)
Telescope + Mount + Guiding: W.O. Star71-ii + iOptron CEM40 EC + Orion Magnificent Mini AutoGuider
Camera: ASI 1600MM Pro + EFW Filter Wheel + Chroma 3nm Siii, Ha, Oiii + ZWO LRGB Filters
Software: PHD2; APT; PixInsight ***** My AP website: www.steveking.pictures
Image
Image
Post Reply

Create an account or sign in to join the discussion

You need to be a member in order to post a reply

Create an account

Not a member? register to join our community
Members can start their own topics & subscribe to topics
It’s free and only takes a minute

Register

Sign in

Return to “Image processing”