ACE Weekly 08/25/2010 - 08/31/2010 All ACE spacecraft subsystems are performing nominally. ======================================================================== Orbit/Attitude: No maneuvers were completed this week. The next attitude maneuver is scheduled for Tuesday 09/07/2010. ======================================================================= OCRs: None ======================================================================== Activities: Data Capture: 100% DOY 234-241 2010 DOY 242 08/30/2010 DSN SPS/TTC Project Interface Test (PIT) DSN requested 2 Project Interface Tests for the upgrade (v9.1.4 -> v10.1) of the DSN Tracking, Telemetry & Command (TTC) system. The first PIT on DOY 235 had used a test SLE CMD Provider. The second PIT on DOY 242 used both the test SLE TLM Provider and the test SLE CMD Provider. At the low data-rate (498bps), telemetry to the MMOC was delayed and came in a single burst. After switching to the high data-rate (87648bps), only 20% of the realtime data arrived in the MMOC and none of the playback data. Also, telemetry from the test SLE TLM Provider was not saved by DSN's Central Data Recorder. 50 minutes into the PIT, DSN switched back to the operational SLE TLM Provider and all telemetry was received in the MMOC. The FOT requested staying with the operational SLE TLM Provider until the last 30 minutes of the PIT. At that time, the spacecraft was switched back to the low data-rate, and the DSN engineers analyzed the equipment while telemetry arrived in the MMOC in bursts every 5 minutes. The data from the first 50 minutes of the pass was redumped on the following pass. Background Information DSN occasionally requests that the missions take part in PITs (Project Interface Test) and Demos. DSN asks that the data during the PITs and Demos be labeled "decommitted". With the limited DSN antenna time and reduced FOT staff, however, we have been using regular tracking passes for PITs and Demos. If the PIT or Demo is unsuccessful, NOAA would lose 3.5 hours of real-time data and the spacecraft's SSR would eventually fill-up and switch to the other SSR (resulting in a minimal 30 second gap). For the vast majority of DSN PITs and Demos, the problems have been minimal and did not impact data. But the PIT on Monday 08/30/2010 had more problems. The loss of data to NOAA and the missing playback data were expected possibilities. The FOT had minimally staffed the PIT and let the automation software run the pass. However, DSN had requested that the FOT switch between the test and operations SLE TLM providers. This was more effort than the FOT had available and the FOT requested that DSN stay with the operational SLE TLM provider and make one last switch to the test SLE TLM provider in the last 30 minutes of the PIT. With the limited DSN antenna time and reduced FOT staff, PITs and Demos will continue to be taken as tracking passes with the MMOC automation software. ======================================================================== Anomalies: DOY 237 08/25/2010 S-ACE-0522 G10-0040 ITOS tm_controller ITOS was unable to process telemetry due to a "timeout reading from tm_controller". ITOS was restarted before beginning of track. This problem occurs approximately once every 2 weeks. DR CDSID#00027515 has been submitted and the ITOS developers expect this to be fixed in the next release of ITOS. IMPACT: Manual intervention required. No delay in activities. DOY 237 08/25/2010 S-ACE-0523 S10-0007 Increase in SBE_CT_B SSR B Single Bit Error Count (SBE_CT_B) increased from 6839 to 7339 since last checked on DOY 231 (08/19/2010). SBE_CT is tracked as a measure of SSR health. IMPACT: Minimal. Single Bit Errors are automatically corrected by the SSR. No non-correctable errors (Hard Single Bit Errors & Double Bit Errors) have occurred throughout the mission. SBE_CT_A SBE_CT_B 1999 15 2000 2001 2002 2003 270 286 2004 858 2005 2006 2118 2007 510 3238 2008 4533 2009 525 6839 2010 780 7339 DOY 235 08/23/2010 S-ACE-0524 S10-0041 Missing 0158 Monitor Data NISN (NASA Integrated Services Network) stopped forwarding DSN's 0158 Monitor Data to the MMOC and SOHO MOC on Monday 08/23/2010 2014z. The FOT noticed the missing data on Thursday 08/26/2010 and contacted NISN. NISN resumed forwarding the monitor data. IMPACT: DSN's Monitor Data is not used by the automation software, but it is used when trouble-shooting problems with DSN. For example, it was used regularly when ACE and WIND had transitioned to the MMOC for trouble-shooting SLE connection problems. DOY 240 08/28/2010 S-ACE-0525 S10-0042 MUS CMD process terminated The MMOC SLE (MUS) cmd process (ace-d24-cmd) terminated unexpectedly 2 minutes after beginning of track. The FOT manually restarted the MUS CMD process. This last occurred on 05/25/2010 (AR#G10-0027). IMPACT: Manual intervention required. Activities delayed 15 minutes. DOY 242 08/30/2010 S-ACE-0526 S10-0043 Problems with TTC V10.1 PIT There were problems using the test SLE TLM Provider during the TTC PIT (Project Interface Test). At low-rate (498bps) telemetry was received in bursts every 5 minutes. At hi-rate (87648bps), only 20% of the realtime data arrived in the MMOC and none of the playback data. 50 minutes into the PIT, DSN switched back to the operational SLE TLM Provider and all telemetry was received in the MMOC. Telemetry from the test SLE TLM Provider was not saved by DSN's Central Data Recorder. DSN does not enter DRs for PITs since the data is considered "decommitted". IMPACT: NOAA was unable to process ~60 minutes of realtime data (50 minutes on test SLE provider + 10 minutes of MMOC reconfiguration). Playback data during this time was redumped on the following pass.