SWPC 1-minute Kp-Index

[Waldo Hazeleger]

The SWPC has a new product (json file).

It is a 1-minute estimated planetary k-index for the last 6 hours. Looks like a really good "real-time" indicator for what the Kp-Index is at the moment. May be a good idea to integrate this on your site (homepage).

Here it is: http://services.swpc.noaa.gov/products/noaa-estimated-planetary-k-index-1-minute.json

 

[Marcel de Bont]

Not really sure if this is a major improvement over what we already have but in the Kp column... what on earth does 3Z, 2Z, 4M etc. mean?

[Waldo Hazeleger]

Well, this new (estimated) Kp is nearly real-time (once every minute). The "official" swpc Kp-Index and any official Kp-Index for that matter can only be calculated after a 3-hour period, due to the way the Kp-Index is to be calculated. See this new product as the swpc equivalent to the postsdam gfz quicklook Kp-index, but then updated every minute instead of once every 15-minutes.

3Z means 3-zero or 3.0, 3P means 3+ or 3.33, 3M means 3- or 2.66

 

[Vancanneyt Sander]

Hi Waldo,

It's always good to look into and compare it against the current Wing-Kp model and see how it performs. I've checked the data in the file and have it processed in line with current Wing Kp:

The greenish line is the 1 minute data and as you see it underperforms the Wing Kp predictions, same counts when comparing observed values:

Mostly quiet conditions currently so we can check back when theres more activity and see how it performs. Currently I don't see much added value, nowcast Kp seems experimental and for the moment, I only trust magnetometers  

[Waldo Hazeleger]

If you only trust magnetometers you should not trust Wing-Kp, since it is solely based on Solar-Wind data. ?

In your graphics, it is clear that the Kp underperforms in the first hour of a new synoptic period. Probably the reason why the GFZ is only reporting the Quicklook Kp for a new synoptic period after the first hour. 

It also interesting to see how this performs when we have more active periods. October 24-27 should be perfect for this.

Edit: How you determine it underperforms? Where do you compare with? It can also be that where you compare with over-estimates.

And by the way: If you want more data, I have the estimated kp data starting from 2016-10-15 04:26:00 (not very much, but a little bit more).

 

 

[Guest Stewart Watt]

Have to admit to never looking at the KP when checking the stats. The aurora is what it is regardless of a number.

Its the Magnetometers for me. When you see the growth phase starting there is usually a glow on the horizon.Then the expansion stage which you see as the aurora "rises' higher in the sky as it expands Southwards.Couple that with a look at what the BZ has been doing over the previous couple of hours and i get more info from that as to what might happen than any KP observed or predicted value. Its just a number to an event as far as i'm concerned.

[Vancanneyt Sander]

Wing Kp is not only derived from solar wind measurements! The input parameters of the Wing Kp +1h is solar wind, IMF and nowcast Kp. The nowcast Kp algoritme at time t can estimate Kp for t -1,5 hour. 

So the data in this 'new' data file from swpc contains the nowcast Kp referred as Kp_est. It is calculated from 9 magnetometer stations in near real-time, but not always all data is gathered of the 9 stations so it can get less accurate if a magnetometer station drops out. It's an automated system that, like other magnetometer stations, use a quiet day curve and calculate the deflections in the readings and convert it to a Kest-indice.

when we look at where the Kp magnetometer stations for Kp estimated and official Potsdam Kp, we see the same mistake like the old Canopus model. All those stations to calculate the nowcast Kp are in the US. So it ain't useful for Europe, China, Australia, ... .

as the wing Kp takes more input parameters besides Kpest and solar wind and IMF, wing still is more reliable. European users are still more with local magnetometer stations as it can defer a lot with USA stations. Of course that is my opinion but open for discussions feedback and more  

[Vancanneyt Sander]

Op 17-10-2016 om 20:43, Waldo Hazeleger zei:

How you determine it underperforms? Where do you compare with? It can also be that where you compare with over-estimates.

It was now solely compared to the live data at that time. Hope you have read my previous post where I mentioned that the nowcast Kp is taken as an input parameter besides solar wind and IMF for the wing Kp predicted algorithm. It's a good thing to compare it with a next storm and see how it performs there. But from what I've read in the science papers about the two, Kpest was proven to be reliable when it was compared with historic official Kp indices, but Wing Kp predicted has more accuracy because it takes also other parameters into account from solar wind and IMF besides the Kpest.

[Vancanneyt Sander]

We are having a storm currently so I've regenerated the data of 1 minute Kp forecast and compared it to wing predicted and wing observed:

The quick look official Kp indices of Potsdam for the 6 to 9 period was 4-, 9-12 was 5- and 12-15 it was 6.

When comparing to Kpest it had two periods where it went down to nothing, that can be the case when it's purely based on magnetometer data where the deflections go up and down. Predicted Kp is mostly in line with the Kpest except in the last few hours where it's predicting bigger storm but last few hours Bz was a bit too much going up and down north/south to intensify storming, Wing Kp does take the solar wind data as an extra to Kpest into account and thus seems more correct, but thats my interpretation love to hear from others too  

[Waldo Hazeleger]

Those dips in Kpest are because Kpest calculation starts again for every 3-hour period. So that influences from a previous are not taken into account for the next period. Disadvantage is that is makes the Kpest less usefull in the first hour of a 3-hour period.

Yesterday we had a storm after the Wind speed at Dscovr started to rise after about 9:40 UTC.  There were more or less two periods of geomagnetic storming. One at 13:50 in the US and one at 22:30 in Europe. The Wing Kp1 and Kpest looked more accurate in the first period than in the second. 

Now I did a little digging on which magnetometers are used in different Kp calculations. I found the following sources: Wing-Kp, Swpc-Kp, Potsdam-Kp.

Wing-Kp is based on: Meanook (CA), Sitka (US), Ottawa (CA), Saint Johns (CA), Newport (US), Fredericksburg (US), Boulder (US), Hartland (UK) and Fresno (US) magnetometers and on Dscovr data.

Swpc-Kp is based on: Sitka (US), Meanook (CA), Ottawa (CA), Frediricksburg (US), Hartland (UK), Wingst (DE), Niemegk (DE) and Canberra (AU) magnetometers.

Potsdam-Kp is based on: Lerwick (UK), Meanook (CA), Sitka (US), Eskdalemuir (UK), Uppsala (SE), Ottawa (CA), Brorfelde (DK), Hartland (UK), Wingst (DE), Niemegk (DE), Frediricksburg (US), Canberra (AU) and Eyrewell (NZ)

Looks to me that advantage of the Wing-Kp is that it also takes dscovr solar-wind data into account, but disadvantage is the (almost) solely US / CA based magnetometers. 

 

[Vancanneyt Sander]

Thanks for the extra input, explains it niveau further! I aggree with your conclusion as well. It's a pity that noaa doesn't use stations from all over the world like Potsdam, maybe they only focus on us customers from the USA :/. This is also the reason why I always liked the SAM magnetometer initiative for its network of personal magnetometers over the world.