ACE Weekly 07/23/04 - 07/29/04

All ACE spacecraft subsystems are performing nominally.  No attitude 
maneuvers were executed this reported period.  The next attitude maneuver 
is scheduled for 08/02/04.

============================================================================
Orbit Events:

 From Craig Roberts (FDF):
The next ACE stationkeeping maneuver (SK-33) is tentatively scheduled for 
08/20/04, to coincide with a (to be) scheduled attitude reorientation 
maneuver.  Thus SK-33 will be out of the way just as the next SEZ 
transit--which will have its closest approach to the Sun on 08/30/04 --is 
beginning.  The delta-V for 08/20/04 is currently projected to be about 
0.37 m/sec in the anti-Sunward direction.  So the burn will be using the 
top-deck axials 1A and 2A.  The burn duration is estimated at 64 seconds.

ACE's Lissajous orbit has now transitioned to an opening pattern, so the 
ACE motion will now be counterclockwise about L1 (and for that matter, the 
Sun) for quite a long time to come.

=====================================================================
OCRs Executed:

NONE

=====================================================================
Ongoing Activities:

TAPS:
- Installation of TAPS release 4.3 is occurred on 07/20/04.  FOT testing of 
this release is ongoing.

----------------------------------------------------------------------------
Automation (ROBOTT):

On Monday 07/26/2004, ROBOTT v0.7 was installed on String 1 and reinstalled
on Strings 2 and 3 (OCR GND-139).  ROBOTT v0.7 has been used for daily
operations since Tuesday 07/27/2004 (OCR GND-140) with ROBOTT in advisory
mode.  No data loss or command problems due to ROBOTT have occurred this
week.

Installation on String 2 had been done on Monday 07/19/2004 and testing was
performed until Monday 07/26/2004.  To date, 14 of the 26 DRs fixed in v0.7
have been verified and 17 new DRs have been written.  A review of operations
with ROBOTT will be held on Friday 07/30/2004.

============================================================================
Extra Spacecraft Activities:

None

=====================================================================
Anomalies:

DOY 04205 (07/23/04) ACE Anomaly G04-0068 - GDS 26m (DSS-16) TCP / 
Commanding Problem
At  12:25:00, the FOT ran the procedure G_PRECAL.  The FOT didn't receive a 
echo for the test block or NO_OP command.  The FOT informed DSS-16 of the 
problem.  DSS-16 stated they had just configured their system and to try 
again.  DSS-16 didn't see the test block or no_op.  After an hour of 
troubleshooting the station dropped their carrier and reloaded their 
equipment.  At 13:25, a NO_OP command was sent and received by the station. 
IMPACT:  Unable to command S/C for an hour.  Delayed pass activities.  SSR 
playback incomplete.  DR# G104650.

DOY 04205 (07/23/04) ACE Anomaly G04-0069 - GDS 26m (DSS-16) C&DH B Not 
Receiving Commands
Commands were correctly received by RcvrA/C&DH A through the spacecraft's 
Hi-Gain Antenna.  But 69 of 70 commands to RcvrB/C&DH B through the 
spacecraft's BroadBeam Antenna were not accepted.  A second support was 
scheduled with DSS-27 to determine if the problem was with the station or 
the spacecraft.  DSS-27 was able to successfully command to RcvrB/C&DH B 
through the BroadBeam Antenna and had no problems.  IMPACT:  Because all of 
the data indicated a nominal uplink (AGC, VCO, CMD Mod Index, CDU Bit 
Lock), the possibility of a spacecraft problem was forwarded to 
management.  If a command was not verified by C&DH B, the RF Watchdog would 
execute 48 hours after the last real-time command and the C&DH would reset 
96 hours after the last poke.  A postpass DR was requested from DSS-16.

DOY 04206 (07/24/04) ACE Anomaly G04-0070 - GDS 34m (DSS-27) Telemetry 
Outage / DCC Problem
At 13:38, the FOT requested DSS-27 put telemetry online.  DSS-27 stated 
data was online, however the FOT was not receiving any data.  At 13:30, the 
FOT informed the TSS of the problem.  At 13:50, the TSS confirmed with JPL 
COMM that data was going to the CDR.  At 13:55, the FOT contacted the CD 
Manager.  The CD Manager performed a down/up  (rebooted) on port 
55550.  However, the CD Manager doesn't see any data coming in from 
JPL.  At 14:05, the FOT informed the TSS that telemetry was still 
offline.  The FOT asked TSS to confirm that data is leaving JPL.  The TSS 
stated that data was going to the CDR, however, they couldn't confirm the 
data was being sent to GSFC.  At 14:15, the TSS stated that DSS-27 was 
switching to a new DCC.  At 14:19, telemetry was 
online.  IMPACT:  Telemetry outage from 13:40 - 14:19.  Pass activities 
were delayed.  The FOT was unable to complete the playback of SSR-B.  The 
remainder of the dump (slices 490 - 601) completed on day 207.  DR# G104653

DOY 04207 (07/25/04) ACE Anomaly S04-0005 - SWICS Limit Violation
TAPS detected a limit violation for SWICS PAPS -5V.  The value was out of 
limits from 207-17:20:52 to 207-17:21:16.  Jim Raines 
notified.  IMPACT:  Unknown.

DOY 04207 (07/25/04) ACE Anomaly G04-0071 - GDS 26m (DSS-16) Command 
Modulation Index Problem
At 12:44:45, telemetry dropped out from DSS-16.  The station was notified 
and telemetry resumed at 12:47:49.  At 12:50:16, during G_INITIAL_CONTACT, 
commanding was automatically paused by TPOCC due to commands not being 
accepted by C&DH-B.  This same problem occurred on day 205 with DSS-16.  At 
12:58, the FOT requested that DSS-16 manually reenter the command 
modulation index.  At 13:00, after reentering the command modulation index, 
a /resume was performed for the procedure G_INITIAL_CONTACT.  At 13:00:30, 
telemetry dropped out again.  The station was informed and telemetry 
resumed at 13:02:34.  The station stated they were reconfiguring their 
system from a test pass that was conducted.  The FOT asked if they should 
hold off commanding until the reconfiguration was complete.  The station 
agreed and stated everything should be complete in 15 minutes. At 13:12, 
DSS-16 gave the FOT a go to resume commanding.  At 13:14, a /resume was 
entered and the command was accepted by C&DH-B.  The pass continued without 
any further problems.  IMPACT:  Pass activities delayed.  DR#G104655.

DOY 04209 (07/27/04) ACE Anomaly S04-0006 - SWICS Limit Violation
TAPS detected 6 limit violation for SWICS PAPS +5V.  The value was out of 
limits from 209-04:06:30 to 209-11:55:55.  Notified Jim Raines.  IMPACT: 
Unknown

DOY 04209 (07/27/04) ACE Anomaly G04-0072 -  ROBOTT Command &Control Failed 
to Start
C&C was unable to allocate enough colors for itself which generated dialog 
boxes.  IMPACT:  C&C would not have started without manual intervention.

DOY 04209 (07/27/04) ACE Anomaly G04-0073 - ROBOTT Paging Not Starting 
After Installation
CRON was unable to start Paging because the file 'page.log' did not 
exist.  IMPACT:  No pages sent to FOT.

DOY 04209 (07/27/04) ACE Anomaly G04-0074 - C&C/ROBOTT Failovers
C&C requests a Failover if it does not see an approved event message within 
14 minutes.  If no event messages are received within 6 minutes, C&C issues 
the TSTOL 'shoval' directive which generates an event message.  However, 
C&C does not issue these 'shoval' directives when it is in Advisory 
Mode.  IMPACT:  Multiple failovers occurred.

DOY 04209 (07/27/04) ACE Anomaly G04-0075 - ROBOTT CRON button Still Green 
After Being Killed
After the third failover today (Ref AR G04-0074), the RTMON issued a 
message stating that there is no system to failover to and to please help. 
At this time cron is also killed. The FOT was unaware that cron was down 
though, because the cron button was still green. The 'crontab -l' directive 
was needed to actually see that cron was down.   IMPACT:  Because the color 
of the button did not change, the FOT was unaware that cron was down until 
none of the postpass activities were performed at EOT + 10 min.

DOY 04210 (07/28/04) ACE Anomaly G04-0076 - GDS 26m (DSS-16) Antenna Problem
At 14:01:33 RCVR A and B dropped out of lock.  DSS-16 lost uplink to 
spacecraft. Had transmitter interlock set with warning: Klystron air flow 
fault detected.  At 14:09:37 RCVR's back in lock and good telemetry 
online.  IMPACT:  2 minutes of degradated data while DSS-16 was bringing 
carrier up.  All data was recovered in playbacks.  DR# G104665

DOY 04211 (07/29/04) ACE Anomaly G04-0077 - Corrupted DSN Schedule
ROBOTT detected a corrupted DSN schedule for Week 32.  FOT was unsure which 
week and day had the corruption because the message from ROBOTT was not 
descriptive enough.  IMPACT:  FOT unable to determine reason for the 
schedule corruption in a timely manner.

DOY 04211 (07/29/04) ACE Anomaly G04-0078 - No Redumps Performed by Automation
S_ops_plbk_completed needs that argument for whether g_end_day has been run 
or not, but this is not updated to a "YES" until after g_end_day 
completes.  So even though g_end_day ran before s_ops_plbk_complete, C&C 
still passed in a "NO" indicating that g_end_day had not run.  IMPACT:  The 
end_day_complete variable is used to determine how much time (20min or 
5min) is needed between redumps and EOT.  There was not enough time to 
complete the redumps with Automation because the argument was not 
correct.  There were 3 telemetry hits during the pass, 2 of which were 
captured on the CDR.  1 hit was not captured which resulted in 6 seconds of 
data loss.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Glen A. Miller
ACE Flight Operations Observatory Engineer
Honeywell Technology Solutions Inc.
Goddard Space Flight Center
Building 14, Room W10A
Mail Stop 428.2
Greenbelt, MD 20771

Office: 301-286-7071
Pager: 800-425-3174
Fax:    301-286-1729
gamiller@pop500.gsfc.nasa.gov
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~