Audio_04_02_2026_15_02_47

Apr 04, 2026 19:57 · 58:34 · English · Whisper Turbo · 8 स्पीकर
आजही हा सण साजरा केला जातो. स्थायी संचयीका करीता अद्ययावत करा →
वाचन अनुक्रमणिकाName
फक्त दर्शविले जात आहे
0:00
S… Speaker 2 (Audio_04_02_2026_15_02_47)
from the ECU and came up with some workaround solution etc and so just because this is a day
0:06
S… Speaker 2 (Audio_04_02_2026_15_02_47)
one ECU many units shipped out um however the on the good good side is from the RD teams have
0:15
S… Speaker 2 (Audio_04_02_2026_15_02_47)
identified workaround solutions for the factory units and also installed units that's good thing
0:22
S… Speaker 2 (Audio_04_02_2026_15_02_47)
so I'll probably hand over to you can you present the ad just uh walk through and then for the team
0:32
S… Speaker 2 (Audio_04_02_2026_15_02_47)
to understand the issue and where we are I try to connect to my laptop thank you can people see the
0:42
S… Speaker 1 (Audio_04_02_2026_15_02_47)
screen yes yes so uh start with the problem description so this issue affects the uh one
0:56
S… Speaker 1 (Audio_04_02_2026_15_02_47)
by nine flex packs which we did a re-spin on last year so it's the gen 4 versions of them that are
1:04
S… Speaker 1 (Audio_04_02_2026_15_02_47)
affected which is uh hardware hardware rail 7 on the LB and 16 on the LA so uh the problem presents itself
1:16
S… Speaker 1 (Audio_04_02_2026_15_02_47)
that the packs can't be upgraded in a sienna shelf without manually intervening so if when the pack
1:25
S… Speaker 1 (Audio_04_02_2026_15_02_47)
is inserted into the shelf if it tries to be upgraded the upgrade will fail and it will remain in that
1:32
S… Speaker 1 (Audio_04_02_2026_15_02_47)
upgrade failed state until uh it is uh cleaned up so uh this issue was was silent for quite a while because
1:45
S… Speaker 1 (Audio_04_02_2026_15_02_47)
the packs don't get upgraded if the shelf is running an older load before rel 16.96
1:52
S… Speaker 1 (Audio_04_02_2026_15_02_47)
uh so uh if it's inserted into a 16.96 shelf nobody will notice that there's a problem because they
2:00
S… Speaker 1 (Audio_04_02_2026_15_02_47)
won't try to upgrade them and then if that shelf then gets upgraded to 16.96 or newer that the upgrade
2:07
S… Speaker 1 (Audio_04_02_2026_15_02_47)
will fail on the pack and that's when uh tiana's customers started noticing this uh two weeks ago
2:16
S… Speaker 1 (Audio_04_02_2026_15_02_47)
so once we established uh the issue we stopped ship on the packs that were in the factory uh and uh
2:25
S… Speaker 1 (Audio_04_02_2026_15_02_47)
they are uh we started applying the corrective action to them so that once uh you've fully confirmed
2:33
S… Speaker 1 (Audio_04_02_2026_15_02_47)
that we'll have actually ship again the root cause is slightly complicated so uh the when the pack boots up
2:43
S… Speaker 1 (Audio_04_02_2026_15_02_47)
it has to determine what the state of the upgrade state machine is in case if the pack had been
2:49
S… Speaker 1 (Audio_04_02_2026_15_02_47)
restarted in the middle of an upgrade we want to be able to continue with the upgrade without any
2:54
S… Speaker 1 (Audio_04_02_2026_15_02_47)
intervention so there's a file persisted on the pack that states what the upgrade state is
3:00
S… Speaker 1 (Audio_04_02_2026_15_02_47)
and if that file is not there there's software that determines what the state should be forces the pack into that
3:09
S… Speaker 1 (Audio_04_02_2026_15_02_47)
state but it seems there's some sort of race condition between the uh provisioning of the pack
3:17
S… Speaker 1 (Audio_04_02_2026_15_02_47)
from the cscnsp and the software that's doing this uh calculation of the state machine state and the
3:29
S… Speaker 1 (Audio_04_02_2026_15_02_47)
state ends up getting determined to be unknown which is not a valid state for the state machine we can't do
3:36
S… Speaker 1 (Audio_04_02_2026_15_02_47)
an upgrade if the state is and there's no way uh for the pack to automatically figure out that it is
3:44
S… Speaker 1 (Audio_04_02_2026_15_02_47)
not in an unknown state if you pull the pack out and put it back in the exact same race condition happens
3:51
S… Speaker 1 (Audio_04_02_2026_15_02_47)
and uh it remains stuck in that state uh if the pack ever does get upgraded everything will be fine the
3:59
S… Speaker 1 (Audio_04_02_2026_15_02_47)
the file will be there saying that it's in the committed state but uh if it's in this unknown state
4:08
S… Speaker 1 (Audio_04_02_2026_15_02_47)
uh it can't recover on its own so the corrective action to recover the pack is uh there we can run a
4:20
S… Speaker 1 (Audio_04_02_2026_15_02_47)
command uh through the the root login of the pack this is this one line command that forces it into the
4:28
S… Speaker 1 (Audio_04_02_2026_15_02_47)
committed state and then once that's done uh it will never be an issue uh on that pack moving forward
4:37
S… Speaker 1 (Audio_04_02_2026_15_02_47)
so that has been tested by kevin and uh and other sienna people on a live customer site uh on monday night
4:46
S… Speaker 1 (Audio_04_02_2026_15_02_47)
i believe and uh it was tested both on the system running 1696 and an older 1690 system so we have a new
4:57
S… Speaker 1 (Audio_04_02_2026_15_02_47)
final test procedure in our factory that
5:00
S… Speaker 3 (Audio_04_02_2026_15_02_47)
adds this command to the final test and that started uh running the packs that are still at
5:07
S… Speaker 3 (Audio_04_02_2026_15_02_47)
the factory through this to to get them all into the valid state and in terms of preventative
5:16
S… Speaker 3 (Audio_04_02_2026_15_02_47)
actions so the this issue was was missed because it doesn't happen if you're not in a sienna shelf
5:23
S… Speaker 3 (Audio_04_02_2026_15_02_47)
if we're testing uh just on a lab bench in the lamentum lab it upgrades with no problem
5:31
S… Speaker 3 (Audio_04_02_2026_15_02_47)
uh and so the older shipping loads that we've had at the very beginning of the program that were on the
5:41
S… Speaker 3 (Audio_04_02_2026_15_02_47)
packs that were sent to sienna for the initial testing of this new hardware did not have this
5:49
S… Speaker 3 (Audio_04_02_2026_15_02_47)
problem so those upgrades in the sienna shelf in the sienna lab worked fine and then an issue was found
5:58
S… Speaker 3 (Audio_04_02_2026_15_02_47)
that required a new shipping load uh late in the program and that was uh the new load was sent to
6:07
S… Speaker 3 (Audio_04_02_2026_15_02_47)
sienna and tested on those packs that they had already but those packs had successfully upgraded
6:12
S… Speaker 3 (Audio_04_02_2026_15_02_47)
so they were not in the same state that a fresh pack that came out of the factory would be in
6:18
S… Speaker 3 (Audio_04_02_2026_15_02_47)
and so the problem wasn't found there and nobody tested a pack with this new shipping load on it in
6:28
S… Speaker 3 (Audio_04_02_2026_15_02_47)
sienna shelf uh nobody tested an upgrade on it uh last year when the this was being rolled out so
6:39
S… Speaker 3 (Audio_04_02_2026_15_02_47)
the the solution is we need to always test upgrades in a sienna uh environment uh so a test cage that's
6:52
S… Speaker 3 (Audio_04_02_2026_15_02_47)
simulating the environment is not sufficient and a sienna shelf that's not configured to carry traffic
6:59
S… Speaker 3 (Audio_04_02_2026_15_02_47)
is insufficient so we do have a shelf in the factory that is used during est testing but it's not
7:07
S… Speaker 3 (Audio_04_02_2026_15_02_47)
configured in any way that's carrying traffic it's just used to power the card and as uh an ethernet
7:15
S… Speaker 3 (Audio_04_02_2026_15_02_47)
source so we uh we don't have a mechanism today in our factory to test cards in a environment that's the
7:24
S… Speaker 1 (Audio_04_02_2026_15_02_47)
same as what a sienna customer would have i have a question um so the workaround can can it be done
7:36
S… Speaker 1 (Audio_04_02_2026_15_02_47)
remotely on our our customer shelves to recover the card yes so kevin you can confirm that but i'm i'm
7:45
S… Speaker 1 (Audio_04_02_2026_15_02_47)
pretty sure there's no uh need to be on site to do this command yes sorry um yes it can be done on uh we
7:56
S… Speaker 2 (Audio_04_02_2026_15_02_47)
have done it on two cards that were running 1696 uh that uh went into upgrade fail state uh because it was
8:08
S… Speaker 2 (Audio_04_02_2026_15_02_47)
being upgraded like kent was saying anything prior to 1696 the car just sits there it runs the shipping
8:15
S… Speaker 2 (Audio_04_02_2026_15_02_47)
load so the problem is dormant until you upgrade 1696 so we have live cards where we rescue uh but had 1696
8:27
S… Speaker 2 (Audio_04_02_2026_15_02_47)
in an upgrade fail state and we successfully upgraded those uh and we also rescue two cards that were live
8:39
S… Speaker 2 (Audio_04_02_2026_15_02_47)
running in pre-1696 you know prior to even while the issue was dormant we rescued those too so we basically
8:47
S… Speaker 2 (Audio_04_02_2026_15_02_47)
put them into a good state so that they would upgrade later to 1696 uh fine and that was confirmed
8:55
S… Speaker 2 (Audio_04_02_2026_15_02_47)
one of those shelves did upgrade uh to 1696 and it was fine so basically we have
9:02
S… Speaker 2 (Audio_04_02_2026_15_02_47)
you know tribal pre-1696 and 1696 upgrade successfully
9:07
S… Speaker 1 (Audio_04_02_2026_15_02_47)
okay so we would be comfortable doing this on like a bigger uh customer base then or do we need to do
9:16
S… Speaker 2 (Audio_04_02_2026_15_02_47)
further testing i don't think we have this problem there's like 3000 plus of these cards that got
9:23
S… Speaker 2 (Audio_04_02_2026_15_02_47)
shipped right so i think we don't have much of a choice but to uh to do sort of a uh card replacement for
9:31
S… Speaker 2 (Audio_04_02_2026_15_02_47)
cars that are sitting in storage at customers and also do live rescues on the ones that are already
9:37
S… Speaker 4 (Audio_04_02_2026_15_02_47)
sitting in shelves right on any release okay so quick question how do we do that for like these
9:44
S… Speaker 4 (Audio_04_02_2026_15_02_47)
3 000 units got shipped to over 50 different customers how do we do that for all these many customers
9:53
S… Speaker 2 (Audio_04_02_2026_15_02_47)
yes so that question i think all right so do we add precedents on how to
10:00
S… Speaker 1 (Audio_04_02_2026_15_02_47)
do this in the with the 80 cars that with the man id problem right yeah so that's the precedence of
10:06
S… Speaker 1 (Audio_04_02_2026_15_02_47)
how we did this so we need to do that on steroid now meaning meaning what again sorry the the 80
10:16
S… Speaker 1 (Audio_04_02_2026_15_02_47)
cars of the same car that we we started uh doing swap right we had the man id issue you're aware
10:23
S… Speaker 5 (Audio_04_02_2026_15_02_47)
of that one yeah yeah i'm aware of that but but that was 80 units and then we are not going to
10:29
S… Speaker 5 (Audio_04_02_2026_15_02_47)
replace this many units in the field are we i don't know well we got we got to talk about this
10:36
S… Speaker 1 (Audio_04_02_2026_15_02_47)
and come to consensus on how we do this uh you know looking at the email i saw from mike uh mccanera
10:46
S… Speaker 1 (Audio_04_02_2026_15_02_47)
this morning we only went through 40 swaps so that's over four months that's only 10 a month so
10:54
S… Speaker 1 (Audio_04_02_2026_15_02_47)
if we have to swap you know 2000 plus cards in storage that will take many years
11:03
S… Speaker 3 (Audio_04_02_2026_15_02_47)
right so how are we going to do this and why do we need yeah go ahead is it a service impacting when
11:11
S… Speaker 3 (Audio_04_02_2026_15_02_47)
you're putting these commands on when you implement the command no it's not service impacting so it
11:18
S… Speaker 3 (Audio_04_02_2026_15_02_47)
doesn't restart the card at all it just you just run the command and then the cards fixed yes okay so
11:24
S… Speaker 2 (Audio_04_02_2026_15_02_47)
can can this be added to uh an upgrade procedure prior to the affected release well that that's this is
11:34
S… Speaker 1 (Audio_04_02_2026_15_02_47)
where i think we're going to call our help from our field support team gcc right we can go and clean
11:40
S… Speaker 1 (Audio_04_02_2026_15_02_47)
up these cards in pre-1696 sounds like an internal discussion yeah some internal discussion we got to
11:48
S… Speaker 1 (Audio_04_02_2026_15_02_47)
execute that you know quickly and at least that narrows down you know less field issues later on when
11:57
S… Speaker 1 (Audio_04_02_2026_15_02_47)
more customer operator 1696 and also with 1695 coming out the door in june right so uh we want to get in and
12:07
S… Speaker 1 (Audio_04_02_2026_15_02_47)
that window before we release another major release in june um i have a question about the preventive
12:17
S… Speaker 2 (Audio_04_02_2026_15_02_47)
action and it's good if momentum only has one shelf uh what release is running on what release will it
12:28
S… Speaker 2 (Audio_04_02_2026_15_02_47)
be this be tested on going forward i think what type of sp or shelf do they have does it matter whether
12:36
S… Speaker 1 (Audio_04_02_2026_15_02_47)
the sp it's it's a it's a hard thing i think it's very difficult i mean it's a conversation to have
12:44
S… Speaker 1 (Audio_04_02_2026_15_02_47)
for sure with momentum uh it's very difficult to maintain uh you know the momentum setup up to date
12:52
S… Speaker 1 (Audio_04_02_2026_15_02_47)
what we need to test is we're going to change well this is what we have in mind and i discussed with
12:57
S… Speaker 1 (Audio_04_02_2026_15_02_47)
ken already and we came to some agreement on it is we're going to change the fpe process we add additional
13:03
S… Speaker 1 (Audio_04_02_2026_15_02_47)
steps that after all fpe tests are done uh we upgrade to a future dummy low which momentum will
13:10
S… Speaker 1 (Audio_04_02_2026_15_02_47)
provide right as part of our fpe qualification so that will guarantee that a production card
13:17
S… Speaker 1 (Audio_04_02_2026_15_02_47)
fresh out of the factory gets an upgrade like the current way we're doing it we're missing that last
13:23
S… Speaker 1 (Audio_04_02_2026_15_02_47)
step we test during development on the beta card but we don't test upgrade when we exit from the factory
13:31
S… Speaker 1 (Audio_04_02_2026_15_02_47)
with a real you know car that came off the the factory line right so uh so we need to change our fpe process
13:42
S… Speaker 2 (Audio_04_02_2026_15_02_47)
that's that would be how are you suggesting that this be done every time shipping load is provided
13:50
S… Speaker 2 (Audio_04_02_2026_15_02_47)
final shipping load is provided yes and then what you're proposing sounds like it would be up to
14:00
S… Speaker 2 (Audio_04_02_2026_15_02_47)
uh someone at sienna to do that testing with the dummy load not momentum yes it's because we have to
14:09
S… Speaker 1 (Audio_04_02_2026_15_02_47)
take the dummy load and build a fake catalog right no momentum wouldn't be able to do that right we have
14:16
S… Speaker 2 (Audio_04_02_2026_15_02_47)
to build fake catalog to a fake future catalog right so okay so it sounds like the preventive action will
14:25
S… Speaker 4 (Audio_04_02_2026_15_02_47)
not happen then the one that's being proposed here it it can't happen at the at momentum it has to be a
14:35
S… Speaker 4 (Audio_04_02_2026_15_02_47)
joint exercise between sienna and momentum so that the upgrade has to take place in a sienna shelf that's
14:42
S… Speaker 4 (Audio_04_02_2026_15_02_47)
configured the way uh a customer system is configured and that's doing that only on the momentum side
14:51
S… Speaker 4 (Audio_04_02_2026_15_02_47)
uh would yeah we definitely can't do it on our own and we would need uh help from someone that's sienna to
14:59
S… Speaker 6 (Audio_04_02_2026_15_02_47)
we would set us
15:00
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Kevin was saying to build a new catalog file for it and that the test can be be done but
15:08
S… Speaker 2 (Audio_04_02_2026_15_02_47)
making sure that the version of software the SP is running is uh is reflective of of the latest
15:15
S… Speaker 2 (Audio_04_02_2026_15_02_47)
versions uh you know this this problem only occurs with certain timing of the provisioning of the
15:25
S… Speaker 2 (Audio_04_02_2026_15_02_47)
pack so there's another pack we did at the same time the RLA 1x5 Gen 4 that has the exact same
15:32
S… Speaker 2 (Audio_04_02_2026_15_02_47)
upgrade engine is a very similar card and it has the same software bug in it but we don't see it
15:39
S… Speaker 2 (Audio_04_02_2026_15_02_47)
because the timing is slightly different when the SP is provisioning a 1x5 versus a 1x9 so to find
15:47
S… Speaker 2 (Audio_04_02_2026_15_02_47)
problems like this we need a test that's it's really going to simulate what uh what's going to
15:53
S… Speaker 2 (Audio_04_02_2026_15_02_47)
happen at the customer sites and that's very difficult for for Lamentum to do on our own
16:00
S… Speaker 3 (Audio_04_02_2026_15_02_47)
so it would need to be a joint activity for an FPE process as a whole right i mean this doesn't just
16:12
S… Speaker 3 (Audio_04_02_2026_15_02_47)
apply to Lamentum it applies to in-house and and other vendors too we don't test a W-O upgrade
16:19
S… Speaker 3 (Audio_04_02_2026_15_02_47)
from a production card anyway this is an internal discussion water right we have i understand but
16:26
S… Speaker 1 (Audio_04_02_2026_15_02_47)
uh lately we have been opting to do virtual FPEs for most of the changes yeah so that uh i mean that
16:36
S… Speaker 1 (Audio_04_02_2026_15_02_47)
has to change for depending on the situation quick question sorry i think i missed that so what what
16:43
S… Speaker 1 (Audio_04_02_2026_15_02_47)
did change that we are seeing this issue and and it wasn't seen before on these products so it's uh
16:57
S… Speaker 2 (Audio_04_02_2026_15_02_47)
it's a race condition that's happening as the card is is moving into the card ready state it is also
17:05
S… Speaker 2 (Audio_04_02_2026_15_02_47)
determining what the upgrade state machine state is and there's a common database that all transactions
17:12
S… Speaker 2 (Audio_04_02_2026_15_02_47)
for provisioning information as well as the upgrade state machine transactions are being put into and
17:18
S… Speaker 2 (Audio_04_02_2026_15_02_47)
there's a a time out in writing to that database when the upgrade state machine is trying to uh initialize
17:25
S… Speaker 2 (Audio_04_02_2026_15_02_47)
the upgrade state to what should be a committed state and because that times out the database never gets
17:31
S… Speaker 2 (Audio_04_02_2026_15_02_47)
updated and it remains in this unknown state which it doesn't work so the uh the problem only happens if you
17:39
S… Speaker 2 (Audio_04_02_2026_15_02_47)
have very specific timing of when the what happened when the card goes card ready and sends the notification
17:47
S… Speaker 2 (Audio_04_02_2026_15_02_47)
to the sp saying it's now ready uh in the in the current load it has not yet done that decision on whether
17:56
S… Speaker 2 (Audio_04_02_2026_15_02_47)
it's committed or unknown and uh if the card is very very busy after that processing provisioning from the sp it
18:06
S… Speaker 2 (Audio_04_02_2026_15_02_47)
is uh is failing to uh in a timely manner uh get that into the database and into a file so if there was
18:16
S… Speaker 2 (Audio_04_02_2026_15_02_47)
you know some change in in how uh provisioning is done this problem might go away as i said on the one
18:24
S… Speaker 2 (Audio_04_02_2026_15_02_47)
by five is the exact same upgrade state machine the exact same sequencing in terms of where it happens
18:31
S… Speaker 2 (Audio_04_02_2026_15_02_47)
relative to where card ready is and we never see this problem on those packs they're on the mdroa packs
18:37
S… Speaker 2 (Audio_04_02_2026_15_02_47)
they're also used the same so uh it's it's something that is uh is hard to reproduce if there's
18:48
S… Speaker 2 (Audio_04_02_2026_15_02_47)
timing changes but in the current configuration it happens 100 of the time so so the the proposed
18:56
S… Speaker 3 (Audio_04_02_2026_15_02_47)
solution here is to change the manufacturing process to commit that database that ken mentioned right
19:01
S… Speaker 3 (Audio_04_02_2026_15_02_47)
so that way we don't care about the timing at all right and so is that part of the corrective action
19:09
S… Speaker 2 (Audio_04_02_2026_15_02_47)
here or preventive action yes yes so uh the very last line of the corrective action so this command
19:18
S… Speaker 2 (Audio_04_02_2026_15_02_47)
we run as part of our new final test so every card that goes through final test after we do the pack
19:24
S… Speaker 2 (Audio_04_02_2026_15_02_47)
cleanup which is erasing a bunch of files we will force this persistent file for the upgrade state
19:30
S… Speaker 2 (Audio_04_02_2026_15_02_47)
machine using this command and then the packs will be safe moving forward so the plan is to do that for
19:40
S… Speaker 2 (Audio_04_02_2026_15_02_47)
all our packs not just this one because i said the one by five and the mdroa have all have the same
19:47
S… Speaker 2 (Audio_04_02_2026_15_02_47)
software bug but it only shows up in the uh today it only shows up in the one by nine but maybe in a
19:54
S… Speaker 2 (Audio_04_02_2026_15_02_47)
year if you know changes something in the recipe it would start showing up on those packs so
19:59
S… Speaker 1 (Audio_04_02_2026_15_02_47)
you know
19:59
S… Speaker 1 (Audio_04_02_2026_15_02_47)
you
20:00
S… Speaker 2 (Audio_04_02_2026_15_02_47)
we will make sure everything we built has that file corrected because the thing is like
20:08
S… Speaker 3 (Audio_04_02_2026_15_02_47)
you see now these units were shipped over a year ago but then we only see the issue reported now
20:16
S… Speaker 3 (Audio_04_02_2026_15_02_47)
after a year so if we only like want to rely on the customer failures that's that's just too late
20:24
S… Speaker 3 (Audio_04_02_2026_15_02_47)
so what can we do to catch such problems before leaving the door yeah so it
20:31
S… Speaker 2 (Audio_04_02_2026_15_02_47)
as Kevin was saying as part of the the FPE process this upgrade needs to be tested before approvals
20:39
S… Speaker 2 (Audio_04_02_2026_15_02_47)
given to start shipping the packs and so today we do a virtual FPE usually on the final shipping
20:50
S… Speaker 2 (Audio_04_02_2026_15_02_47)
load and so there's no card that leaves our factory that goes to Siena with that final
20:56
S… Speaker 2 (Audio_04_02_2026_15_02_47)
shipping load is Siena already has packs that were had older loads on them and and those
21:04
S… Speaker 2 (Audio_04_02_2026_15_02_47)
older loads had no upgrade problems so this was great but moving forward we need to be sure
21:09
S… Speaker 2 (Audio_04_02_2026_15_02_47)
for any new new hardware LCM activity that as part of the FPE process we we do this upgrade
21:19
S… Speaker 3 (Audio_04_02_2026_15_02_47)
so if it's not documented here I think we need to to document with what you just mentioned right
21:27
S… Speaker 2 (Audio_04_02_2026_15_02_47)
that's an action to take yeah yeah that's for for new car development right yeah so yeah so this here
21:38
S… Speaker 2 (Audio_04_02_2026_15_02_47)
is is is the statement it's uh where where that's done uh is could logical place to to do it is uh is at Siena in a shelf that uh that is running the right environment
21:56
S… Speaker 3 (Audio_04_02_2026_15_02_47)
yeah yeah the reason I asked like I'm just wondering what what documents are going to be revised
22:03
S… Speaker 3 (Audio_04_02_2026_15_02_47)
to to reflect on this issue like is it is it our like software qualification plans or is it the NPI exit criteria
22:14
S… Speaker 1 (Audio_04_02_2026_15_02_47)
release checks what are we going to change publication I would say we don't we release the card
22:30
S… Speaker 1 (Audio_04_02_2026_15_02_47)
production so there's going to be a check box that says how we test future upgrade right okay sounds
22:39
S… Speaker 1 (Audio_04_02_2026_15_02_47)
good and and who has that action I think that falls on Walter you guys own the qualification process
22:51
S… Speaker 1 (Audio_04_02_2026_15_02_47)
for uh new car development no no then who we do the we could put it in the load that you're talking
23:00
S… Speaker 1 (Audio_04_02_2026_15_02_47)
about right no but the final the final approval we report the software FPE testing to
23:08
S… Speaker 1 (Audio_04_02_2026_15_02_47)
you guys and then Shala or someone in your team would release the final qualification right
23:15
S… Speaker 1 (Audio_04_02_2026_15_02_47)
we could put it as part of the FPE as one test case test to dummy load right I could do it there
23:22
S… Speaker 1 (Audio_04_02_2026_15_02_47)
that's fine too but uh if you really want something formal that uh is in the qualification report
23:30
S… Speaker 1 (Audio_04_02_2026_15_02_47)
then you you want it and you and your I think the upgrade to uh to uh dummy load would cover it
23:38
S… Speaker 1 (Audio_04_02_2026_15_02_47)
okay I'm fine with that too but the thing is we need to understand from Lumentum uh what is the
23:56
S… Speaker 1 (Audio_04_02_2026_15_02_47)
permanent solution and I it sounds like that command that will be added will prevent this in the future
24:05
S… Speaker 2 (Audio_04_02_2026_15_02_47)
for any new load yes this this particular problem uh the the final test software that is being updated for
24:15
S… Speaker 2 (Audio_04_02_2026_15_02_47)
all our packs will will cover off this particular problem but the concern would be you know this
24:22
S… Speaker 2 (Audio_04_02_2026_15_02_47)
is just one type of problem that could occur that could cause an upgrade to fail and and we we need to
24:28
S… Speaker 2 (Audio_04_02_2026_15_02_47)
do it an exhaustive test as well other than this fixing this one specific issue yeah yeah and I think we need to agree on
24:59
S… Speaker 1 (Audio_04_02_2026_15_02_47)
to win it
24:59
S… Speaker 1 (Audio_04_02_2026_15_02_47)
to win it
25:00
S… Speaker 1 (Audio_04_02_2026_15_02_47)
virtually yes yeah so that that area yeah we have to talk about that right yeah so
25:19
S… Speaker 1 (Audio_04_02_2026_15_02_47)
there's two problems here how to deal with the future problem right but
25:22
S… Speaker 1 (Audio_04_02_2026_15_02_47)
there's also we haven't talked much here about how we deal with the current
25:26
S… Speaker 2 (Audio_04_02_2026_15_02_47)
problem how we're gonna roll this out right yeah that's the thing so can we
25:30
S… Speaker 2 (Audio_04_02_2026_15_02_47)
get a picture of what's the process now if you want to pick one customer like
25:36
S… Speaker 2 (Audio_04_02_2026_15_02_47)
what's the process of to fixing this issue in the field well there's there's
25:44
S… Speaker 1 (Audio_04_02_2026_15_02_47)
there's several things that are up in flight right now right momentum is going
25:49
S… Speaker 1 (Audio_04_02_2026_15_02_47)
to restart production you see here on April 7th or 8th with the new method right
26:00
S… Speaker 1 (Audio_04_02_2026_15_02_47)
the new manufacturing procedure where Ken mentioned he's going to commit the
26:04
S… Speaker 1 (Audio_04_02_2026_15_02_47)
database that will prevent any kind of race condition to happen to cause
26:10
S… Speaker 1 (Audio_04_02_2026_15_02_47)
any issue so that we're not dependent on SP type or you know any kind of weird
26:16
S… Speaker 1 (Audio_04_02_2026_15_02_47)
timing between SP or the car to trigger this issue right so that's that's the
26:22
S… Speaker 1 (Audio_04_02_2026_15_02_47)
solution for the current cards and so that production will start on the 7th to 8th
26:30
S… Speaker 1 (Audio_04_02_2026_15_02_47)
you know after momentum confirm all the do all the quality checks for a new car
26:35
S… Speaker 1 (Audio_04_02_2026_15_02_47)
using this procedure a new car coming off the factory line and the second thing we
26:42
S… Speaker 1 (Audio_04_02_2026_15_02_47)
need to deal with now is there's several categories of cards in the field that have
26:48
S… Speaker 1 (Audio_04_02_2026_15_02_47)
this problem there are the ones sitting in storage rooms those ideally we should
26:53
S… Speaker 1 (Audio_04_02_2026_15_02_47)
recall them but you know given the speed of the recall I don't know how we're
26:58
S… Speaker 1 (Audio_04_02_2026_15_02_47)
going to handle that there's possibly 2,000 plus cards sitting in storage okay the
27:06
S… Speaker 1 (Audio_04_02_2026_15_02_47)
second category of cards are the ones that are plugged into 1696 shelf and those
27:12
S… Speaker 1 (Audio_04_02_2026_15_02_47)
will obviously fail with upgrade failure and I GCC already has our recovery
27:17
S… Speaker 1 (Audio_04_02_2026_15_02_47)
procedure documented on how to recover those so that I think it's taken care of so
27:25
S… Speaker 1 (Audio_04_02_2026_15_02_47)
far there's only two incidents one at UNITEL the other one at Vodafone okay sorry good
27:32
S… Speaker 2 (Audio_04_02_2026_15_02_47)
question so you mentioned GCC knows how to fix that so that's including this new command or that
27:39
S… Speaker 1 (Audio_04_02_2026_15_02_47)
that yes the command that is implemented in manufacturing is the same as the one in the
27:46
S… Speaker 1 (Audio_04_02_2026_15_02_47)
recovery procedure we published GCC but you know the one we publish to GCC has our other steps
27:53
S… Speaker 1 (Audio_04_02_2026_15_02_47)
you know to cleanly do this like because it's a live network and because it's a it's a it's in
27:59
S… Speaker 1 (Audio_04_02_2026_15_02_47)
our shelf momentum doesn't have a view of how it works in the shelf right so they're they're
28:04
S… Speaker 1 (Audio_04_02_2026_15_02_47)
focused on on just the command at their level on the card level but we have steps very well documented
28:12
S… Speaker 1 (Audio_04_02_2026_15_02_47)
steps and we have executed as I said earlier on four cards already in the field all right following
28:19
S… Speaker 1 (Audio_04_02_2026_15_02_47)
those steps so that that category is taken care of that should be just a small number of cards right
28:25
S… Speaker 1 (Audio_04_02_2026_15_02_47)
now because there's not too many people in 1696 yet and there's probably a much bigger category of
28:34
S… Speaker 1 (Audio_04_02_2026_15_02_47)
cards that are inserted in shelves that are pre-1696 with the problem dormant so you know my proposal here
28:44
S… Speaker 1 (Audio_04_02_2026_15_02_47)
uh is that we uh we we have to mobilize GCC to go find all these cards do the recovery on them pre the
28:54
S… Speaker 1 (Audio_04_02_2026_15_02_47)
up pre doing the upgrade to 1696 or or 175 in June uh you know so that they don't even see any problem on
29:04
S… Speaker 1 (Audio_04_02_2026_15_02_47)
that upgrade that's right that's what i'm saying that we added to a pre-upgrade checklist to see if
29:11
S… Speaker 1 (Audio_04_02_2026_15_02_47)
there are any of those cards there and then do that those are for the deploy ones running in pre 1696
29:21
S… Speaker 1 (Audio_04_02_2026_15_02_47)
yeah so that's that's i think that's how we should attack this problem right so there's probably be a
29:29
S… Speaker 1 (Audio_04_02_2026_15_02_47)
huge amount a number of cards that are sitting in storage let's say 2 000 and let's say maybe
29:37
S… Speaker 1 (Audio_04_02_2026_15_02_47)
10 uh hundreds or maybe over five six hundred or something possibly inserted into shelves
29:46
S… Speaker 1 (Audio_04_02_2026_15_02_47)
uh running free 1696 and probably uh tens of ones that are running 1696 because i only seen two
29:55
S… Speaker 1 (Audio_04_02_2026_15_02_47)
report so i'm assuming that's just a smaller set
30:00
S… Speaker 4 (Audio_04_02_2026_15_02_47)
right so we gotta divine conquer here right different teams have to be involved
30:12
S… Speaker 8 (Audio_04_02_2026_15_02_47)
yeah so i think that that is an internal meeting kevin i think here we wanted to understand what
30:22
S… Speaker 8 (Audio_04_02_2026_15_02_47)
the problem is it sounds like that is the corrective action i'm still not clear on how
30:38
S… Speaker 1 (Audio_04_02_2026_15_02_47)
the preventative action whether that would be enough or yeah but you know we got to deal with
30:46
S… Speaker 4 (Audio_04_02_2026_15_02_47)
the current situation first right there's a whole bunch of you know three thousand plus car i don't
30:53
S… Speaker 4 (Audio_04_02_2026_15_02_47)
see i don't know if you've seen the numbers it's like three thousand plus cards that have been
30:56
S… Speaker 1 (Audio_04_02_2026_15_02_47)
shipped since may last year right that is there and that's what i'm saying that we need an internal
31:03
S… Speaker 1 (Audio_04_02_2026_15_02_47)
meeting for that how we deal with that oh you know usually quality here and uh momentum quality
31:11
S… Speaker 4 (Audio_04_02_2026_15_02_47)
and our quality team does the recall so i guess you guys have to talk that one out i don't know how
31:18
S… Speaker 2 (Audio_04_02_2026_15_02_47)
we haven't decided to do a recall so that's that's what i guess walter is is suggesting that we need
31:26
S… Speaker 2 (Audio_04_02_2026_15_02_47)
to have the proposals and team come together to make a decision if you actually want to recall right
31:33
S… Speaker 5 (Audio_04_02_2026_15_02_47)
yeah i don't think it's cost effective to recall these cards if we can fix it remotely and it's
31:39
S… Speaker 1 (Audio_04_02_2026_15_02_47)
not service affecting yeah that's a good good answer too yeah give in give in every set of that fixes
31:46
S… Speaker 1 (Audio_04_02_2026_15_02_47)
units that are plugged in but that doesn't address units that you need to plug in right agree but could
31:53
S… Speaker 5 (Audio_04_02_2026_15_02_47)
we put ask them to put in a lab and then do those upgrades or do the fix in a lab shelf the problem is
32:01
S… Speaker 4 (Audio_04_02_2026_15_02_47)
some customers don't have an empty shelf to uh but who's going to do it right does the cuffs are do
32:07
S… Speaker 4 (Audio_04_02_2026_15_02_47)
it or or are a few support guys do it we gotta have that conversation right we can i don't think we
32:15
S… Speaker 1 (Audio_04_02_2026_15_02_47)
expect the customer to do this and so that's why we can understand where the warehouses are could
32:21
S… Speaker 8 (Audio_04_02_2026_15_02_47)
there be a shelf nearby where they could be like with the help of the vendor provide a resource to
32:30
S… Speaker 1 (Audio_04_02_2026_15_02_47)
upgrade and that type of thing right well wilhelm is closest to the field support level yeah sorry kevin
32:36
S… Speaker 3 (Audio_04_02_2026_15_02_47)
yeah just a comment right customers aren't going to be able to do this because we need some special
32:39
S… Speaker 3 (Audio_04_02_2026_15_02_47)
passwords right to be able to go into a group on the on the card so there's no way we're gonna
32:44
S… Speaker 3 (Audio_04_02_2026_15_02_47)
be able to let customers do this at all right so take that off the table completely and it's going to
32:49
S… Speaker 3 (Audio_04_02_2026_15_02_47)
be a massive coordination effort here to try to figure out where all these cards are and try to
32:56
S… Speaker 3 (Audio_04_02_2026_15_02_47)
do something ahead of time we can also add in pre-upgrade checks to going to 1696 or later releases
33:04
S… Speaker 3 (Audio_04_02_2026_15_02_47)
to try to catch those there but not all customers execute those um so we need to have a good view
33:11
S… Speaker 3 (Audio_04_02_2026_15_02_47)
of where do all these cards go who's got the vast majority of them and and have some plans put in
33:17
S… Speaker 3 (Audio_04_02_2026_15_02_47)
place like on a per customer basis of what are we going to do to try to recover their cards and this
33:22
S… Speaker 3 (Audio_04_02_2026_15_02_47)
is going to take a lot of time and a lot of effort to coordinate all this
33:25
S… Speaker 7 (Audio_04_02_2026_15_02_47)
so saeed at this this stage do you know how many cards not left sienna yet like just a high level how
33:34
S… Speaker 7 (Audio_04_02_2026_15_02_47)
many still in sienna how many already left sienna like went to any customer so of the of the 3500 units
33:46
S… Speaker 2 (Audio_04_02_2026_15_02_47)
yeah it looks like we have not shipped 700 of those we have 250 in our inventory but the remaining of
34:02
S… Speaker 2 (Audio_04_02_2026_15_02_47)
that 700 minus 255 i don't know where where those units are it doesn't show in our system it shows
34:11
S… Speaker 2 (Audio_04_02_2026_15_02_47)
it's lumentum canada so we need to figure out where those units are but then 2800 units are shipped to
34:23
S… Speaker 2 (Audio_04_02_2026_15_02_47)
over 50 or even more customers see 2000 and how many ship to customers 800 yeah it's it's beyond 50
34:37
S… Speaker 2 (Audio_04_02_2026_15_02_47)
50. there are like probably around yeah it's beyond 50 i mean there are some that let's say it was shipped
34:48
S… Speaker 2 (Audio_04_02_2026_15_02_47)
to tara but different uh taras but but we have customers who received eight seven six units but we have
34:59
S… Speaker 6 (Audio_04_02_2026_15_02_47)
the top
34:59
S… Speaker 1 (Audio_04_02_2026_15_02_47)
the top
35:00
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Top like ones that have like four hundred or like hundred or sixty units
35:08
S… Speaker 2 (Audio_04_02_2026_15_02_47)
There is maybe I quickly share my screen so you can actually see it
35:12
S… Speaker 1 (Audio_04_02_2026_15_02_47)
But to Kevin's comment you believe most of that is actually not installed base. It's it's just in warehousing. It's pending. It's
35:23
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Not in a shelf yet. I can't tell that it's it's not in a shelf running 1696
35:30
S… Speaker 1 (Audio_04_02_2026_15_02_47)
It could be a storage. It's it's just working but not necessarily gotcha
35:37
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Yeah, we I don't know how we can get a breakdown
35:40
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Would have the information on how many are sitting storage and
35:46
S… Speaker 4 (Audio_04_02_2026_15_02_47)
waiting a plug-in
35:48
S… Speaker 1 (Audio_04_02_2026_15_02_47)
We'd have to ask for all the customer inventories
35:52
S… Speaker 2 (Audio_04_02_2026_15_02_47)
And you see this long list it's it's it's at this time over a hundred
36:02
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Customer name attached to the blank that means those ones are like either in Siena inventory or somewhere
36:09
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Not to the customers right that that that's correct
36:13
S… Speaker 2 (Audio_04_02_2026_15_02_47)
So those blanks ones like you see
36:16
S… Speaker 2 (Audio_04_02_2026_15_02_47)
It it's it's spread between what says like Lumentum Canada and somewhere within Siena
36:27
S… Speaker 2 (Audio_04_02_2026_15_02_47)
So there is like
36:30
S… Speaker 2 (Audio_04_02_2026_15_02_47)
For 400 units. It says Lumentum Canada
36:36
S… Speaker 2 (Audio_04_02_2026_15_02_47)
I'm not sure where are those?
36:43
S… Speaker 2 (Audio_04_02_2026_15_02_47)
But yeah, so 700 not yet shipped to to end customers
36:49
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Around 2800 out there. Yep
37:03
S… Speaker 2 (Audio_04_02_2026_15_02_47)
And yeah, there you see the breakdown by customers just so many
37:08
S… Speaker 4 (Audio_04_02_2026_15_02_47)
I think well, I think we need GCC right with a clean-up effort. Well, definitely right?
37:24
S… Speaker 3 (Audio_04_02_2026_15_02_47)
But this is gonna be a massive under
37:26
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Yeah, from this call. I'm hearing that
37:33
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Shipping these cars back we're not a solution either, right?
37:40
S… Speaker 3 (Audio_04_02_2026_15_02_47)
Well, it sounds like there might be 700 right to deal with as a separate problem
37:44
S… Speaker 3 (Audio_04_02_2026_15_02_47)
They haven't been shipped yet. Like someone's got to look at what's the most efficient way to deal with those
37:51
S… Speaker 3 (Audio_04_02_2026_15_02_47)
Versus the ones that have actually been shipped
37:53
S… Speaker 5 (Audio_04_02_2026_15_02_47)
Yeah
37:54
S… Speaker 2 (Audio_04_02_2026_15_02_47)
So again, like from
37:58
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Like is end of quarter and and I don't know if we want to jeopardize revenue by purging and returning all these units
38:07
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Again, we need to have that discussion internally, but we could just
38:14
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Perhaps let these go and then we deal it the same way we are dealing with the other
38:20
S… Speaker 2 (Audio_04_02_2026_15_02_47)
3000 units, right?
38:29
S… Speaker 3 (Audio_04_02_2026_15_02_47)
More involved in just trying to deal with ones that are in hand. We haven't sent it already
38:33
S… Speaker 5 (Audio_04_02_2026_15_02_47)
Yeah
38:34
S… Speaker 3 (Audio_04_02_2026_15_02_47)
So someone should do that compare and contrast first I would think
38:37
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Because dealing with them in the field needs a maintenance window and needs GCC engagement with the customer, right?
38:44
S… Speaker 4 (Audio_04_02_2026_15_02_47)
So it's uh, it's also equally uh
38:48
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Unconsuming for the customer, too
38:51
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Right
38:51
S… Speaker 2 (Audio_04_02_2026_15_02_47)
So Kevin, I don't know Kevin or or will have can we get the proposals of what what we think?
38:59
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Are the path forward and then have a call to discuss that with GCC and whoever else needs to be involved?
39:05
S… Speaker 3 (Audio_04_02_2026_15_02_47)
Certainly bring them into it. I'd be interested to know what's the plan to deal with the 700 haven't been shipped yet
39:19
S… Speaker 3 (Audio_04_02_2026_15_02_47)
Yeah, so clearly we got to involve the GCC team on what's been deployed to field and what's it going to take
39:24
S… Speaker 3 (Audio_04_02_2026_15_02_47)
Um, you know in that maintenance window last Thursday night to recover those cards there
39:29
S… Speaker 3 (Audio_04_02_2026_15_02_47)
We're talking 20 to 30 minutes
39:32
S… Speaker 3 (Audio_04_02_2026_15_02_47)
To go do that that's a the time to go in and execute run it do the upgrade make sure things are good
39:38
S… Speaker 3 (Audio_04_02_2026_15_02_47)
Um, you know if we add that up multiply let's say 30 minutes by the number of cards that have to be done
39:43
S… Speaker 3 (Audio_04_02_2026_15_02_47)
That's a lot of hours and a lot of coordination you need to get to get to that
39:51
S… Speaker 6 (Audio_04_02_2026_15_02_47)
So we should see what the current uh requirement is for end of quarter
39:58
S… Speaker 6 (Audio_04_02_2026_15_02_47)
We can make to set that decision set
40:00
S… Speaker 5 (Audio_04_02_2026_15_02_47)
really and if there is surplus deal with that differently i think there's this three groupings
40:05
S… Speaker 4 (Audio_04_02_2026_15_02_47)
really there's the one the 700 sitting in our shop which we can swap for good ones
40:10
S… Speaker 4 (Audio_04_02_2026_15_02_47)
right that's under our control there's a second group that are in storage room at customers
40:17
S… Speaker 4 (Audio_04_02_2026_15_02_47)
which you know they're in storage rooms so gcc wouldn't be involved in those so we could
40:23
S… Speaker 4 (Audio_04_02_2026_15_02_47)
you know swap those too and they're the ones that are already put into shelves doesn't matter which
40:31
S… Speaker 4 (Audio_04_02_2026_15_02_47)
release those ones gcc could attack and clean up right so we we kind of need a breakdown on
40:38
S… Speaker 4 (Audio_04_02_2026_15_02_47)
how many are inserted i think then we can do the the math on uh man hours but does lumen even have
40:47
S… Speaker 5 (Audio_04_02_2026_15_02_47)
proactive units for us to swap if we were to take that option right does lumentum have 700 units
40:53
S… Speaker 1 (Audio_04_02_2026_15_02_47)
staying around ready with not 700 but we can check our inventory and we can get back to you pretty
40:59
S… Speaker 1 (Audio_04_02_2026_15_02_47)
quickly on that point so we'll see what we can offer in terms of uh spares also with my limited knowledge
41:11
S… Speaker 6 (Audio_04_02_2026_15_02_47)
we are like the solutions that can mention it's already uh tested and then we should be by monday
41:18
S… Speaker 6 (Audio_04_02_2026_15_02_47)
or tomorrow monday we'll be uh remove the ship hold that we can we can move forward with testing
41:24
S… Speaker 6 (Audio_04_02_2026_15_02_47)
sounds like it doesn't take a long time to test retest this card right kevin if we say you need to
41:32
S… Speaker 6 (Audio_04_02_2026_15_02_47)
return back to us to rework at least fast right yeah it should be fast yeah it's just a matter of running
41:42
S… Speaker 2 (Audio_04_02_2026_15_02_47)
it through the final test again which is uh provided they're new in box right and that they're not in
41:49
S… Speaker 1 (Audio_04_02_2026_15_02_47)
the shelf and have been used and whatnot yes so first step i mean let's go figure out what we can offer
41:55
S… Speaker 1 (Audio_04_02_2026_15_02_47)
because if you take something out of the system you gotta put something back in um so you don't run dry
42:00
S… Speaker 1 (Audio_04_02_2026_15_02_47)
so that's that's fair we'll uh let's look at that one and figure out how do we get clean units starting
42:05
S… Speaker 1 (Audio_04_02_2026_15_02_47)
to flow in and then we'll have to figure out how to swap back um what's out there right yeah so if
42:19
S… Speaker 7 (Audio_04_02_2026_15_02_47)
we send those units if we can get like units for them and also what's the turnaround time
42:29
S… Speaker 6 (Audio_04_02_2026_15_02_47)
to get them fixed so see the cue share the file after the meeting so for those 700 you see the many
42:38
S… Speaker 6 (Audio_04_02_2026_15_02_47)
list is the momentum right the momentum of the canada or something yeah we have to base on serial number
42:45
S… Speaker 3 (Audio_04_02_2026_15_02_47)
to search where are those cards yeah for sure i'll send that yeah this this breakdown here is for
42:54
S… Speaker 7 (Audio_04_02_2026_15_02_47)
both specs yes both the la or yeah both both of them the lb and la of the la though the top line is
43:13
S… Speaker 3 (Audio_04_02_2026_15_02_47)
three quarters right maybe that you've given a higher priority well yeah the lb i think a more
43:26
S… Speaker 6 (Audio_04_02_2026_15_02_47)
popular flex uh shift the units we have a 322 la 2163 lb more lb than la for total shift field
43:42
S… Speaker 5 (Audio_04_02_2026_15_02_47)
population is about the same just a little bit more lb but it's not that drastically different
43:48
S… Speaker 4 (Audio_04_02_2026_15_02_47)
okay i think the la is for uh fixed grid use right or we label it as fixed grid that's the one yeah okay
44:00
S… Speaker 4 (Audio_04_02_2026_15_02_47)
that means we've got a lot of customers still on fixed grid most are buying flex yeah and ken a
44:10
S… Speaker 3 (Audio_04_02_2026_15_02_47)
question for you you indicated that the 553 r a b by 5 is not prone to this issue because of timing
44:26
S… Speaker 3 (Audio_04_02_2026_15_02_47)
differences yes but you said that if something changes on the sp side they it could also happen
44:36
S… Speaker 2 (Audio_04_02_2026_15_02_47)
yeah it's theoretically we would need to do some more analysis and we we're still working on a software
44:45
S… Speaker 2 (Audio_04_02_2026_15_02_47)
fix uh for addressing the problem in uh in the future load but so we we haven't quite gotten the
44:56
S… Speaker 2 (Audio_04_02_2026_15_02_47)
the details of what's required around the timing how big is the window and that's
45:00
S… Speaker 4 (Audio_04_02_2026_15_02_47)
things so so this is a latent issue on the on that peg as well but to a lesser degree
45:07
S… Speaker 2 (Audio_04_02_2026_15_02_47)
yes it is potentially uh an issue uh but it that's we're not ready at this point to
45:20
S… Speaker 2 (Audio_04_02_2026_15_02_47)
to say what the likelihood is of this happening if uh if there's minor changes in how the sp is
45:28
S… Speaker 2 (Audio_04_02_2026_15_02_47)
is talking to the current it may be that the window isn't isn't really open at all on those
45:34
S… Speaker 2 (Audio_04_02_2026_15_02_47)
cards just because of of how quickly stuff happens but i mean the volume started much more yeah i i
45:43
S… Speaker 1 (Audio_04_02_2026_15_02_47)
have no idea about volumes to be honest but uh but we're gonna change the manufacturing process for
45:48
S… Speaker 1 (Audio_04_02_2026_15_02_47)
all gen 4 cards right ken yeah so newt so newly manufactured uh gen 4 cards will have the fixed
45:55
S… Speaker 2 (Audio_04_02_2026_15_02_47)
run on them the the question is you know gen for one by fives that are are in boxes do we need to
46:02
S… Speaker 2 (Audio_04_02_2026_15_02_47)
do anything about them and uh without a bit more information i i don't think we can answer that
46:09
S… Speaker 2 (Audio_04_02_2026_15_02_47)
question but hopefully by the middle of next week when we've done more analysis of the of the root
46:16
S… Speaker 1 (Audio_04_02_2026_15_02_47)
cause of the i think you should eliminate that um yeah but walter we haven't seen any incident yet so
46:25
S… Speaker 1 (Audio_04_02_2026_15_02_47)
i i agree with you it is a concern but uh we haven't seen any incident yet and this problem is only
46:34
S… Speaker 1 (Audio_04_02_2026_15_02_47)
there on the first upgrade so if six if one by five out there gets upgraded already that means they're
46:41
S… Speaker 1 (Audio_04_02_2026_15_02_47)
good so i think the window is even smaller right well if they get through one upgrade already no but
46:51
S… Speaker 4 (Audio_04_02_2026_15_02_47)
didn't you say that that this command will be done on every new unit yes only on new unit on the gen 4s
47:04
S… Speaker 1 (Audio_04_02_2026_15_02_47)
that's right but right but the current rla 1x5 out there which are gen 4 right the new the new lcm ones
47:12
S… Speaker 1 (Audio_04_02_2026_15_02_47)
from last year we haven't seen a problem with those strangely it's because of the timing that ken
47:20
S… Speaker 4 (Audio_04_02_2026_15_02_47)
invention there i understand that but he said that uh so if something changes on the sp it could happen
47:27
S… Speaker 4 (Audio_04_02_2026_15_02_47)
so what why don't not try to avoid this from the new production yeah just try to
47:36
S… Speaker 2 (Audio_04_02_2026_15_02_47)
yeah so we are we are addressing it for any new production uh you know i think obviously the the
47:44
S… Speaker 2 (Audio_04_02_2026_15_02_47)
priority is is dealing with the one by nines because we know that that's an issue and once we have a
47:49
S… Speaker 2 (Audio_04_02_2026_15_02_47)
better handle on on what the risk is to the one by flies and we can decide do we need a similar
47:56
S… Speaker 2 (Audio_04_02_2026_15_02_47)
program to uh to address cards that are still in boxes of one by fives and you know most likely i
48:06
S… Speaker 2 (Audio_04_02_2026_15_02_47)
think there we can we may be able to prove that it's not going to be ever be an issue for the one by
48:13
S… Speaker 2 (Audio_04_02_2026_15_02_47)
five but we're we're not going to do that yet so i i think uh you know in in a week or two we may
48:20
S… Speaker 2 (Audio_04_02_2026_15_02_47)
decide okay after the one by nines are taken care of we may need to do something similar sort of one
48:25
S… Speaker 2 (Audio_04_02_2026_15_02_47)
by fives but i think with another week of of investigation by the software team we'll have a
48:32
S… Speaker 3 (Audio_04_02_2026_15_02_47)
better handle on on what the real risk is the one by fives okay you mentioned the full manufacturing for
48:38
S… Speaker 3 (Audio_04_02_2026_15_02_47)
one by five is covered right we're going to have this new yes yes the manufacturing test will release
48:45
S… Speaker 2 (Audio_04_02_2026_15_02_47)
new final test software for all packs on that platform including one by five right yeah including
48:53
S… Speaker 3 (Audio_04_02_2026_15_02_47)
one by five including md roa okay i just did some note here uh we will find the bar will check our
49:14
S… Speaker 3 (Audio_04_02_2026_15_02_47)
spares for la and b lb just see you know if sienna wants to get some cards out of the uh field uh what
49:23
S… Speaker 5 (Audio_04_02_2026_15_02_47)
can we support and say you will share this linda i i don't think those are for the field units but for
49:29
S… Speaker 5 (Audio_04_02_2026_15_02_47)
the one that are still not shipped so for the 700 units yeah for the yes we are going to check spares
49:38
S… Speaker 3 (Audio_04_02_2026_15_02_47)
and really based on sienna's needs or whatever you want to us to send right to see we can see what we can
49:44
S… Speaker 3 (Audio_04_02_2026_15_02_47)
send for for the ones not shipped maybe it's relatively easy to rework those i'm just thinking
49:52
S… Speaker 3 (Audio_04_02_2026_15_02_47)
based on kevin the the rework from our side you know manufacturing is it is a faster rework
50:01
S… Speaker 1 (Audio_04_02_2026_15_02_47)
So if it's now shipped, maybe it can be done fast, right?
50:09
S… Speaker 2 (Audio_04_02_2026_15_02_47)
We'd assume that some of these you'd ship back,
50:11
S… Speaker 2 (Audio_04_02_2026_15_02_47)
but in order to go and maintain your, you know, quarter-end shipping,
50:15
S… Speaker 2 (Audio_04_02_2026_15_02_47)
we need to go and provide up information,
50:17
S… Speaker 2 (Audio_04_02_2026_15_02_47)
what we can provide a short order that they could use
50:20
S… Speaker 2 (Audio_04_02_2026_15_02_47)
to go and meet their customer ship requirements
50:22
S… Speaker 2 (Audio_04_02_2026_15_02_47)
and then cycle back things.
50:24
S… Speaker 2 (Audio_04_02_2026_15_02_47)
So, I mean, there has to be an exchange of parts
50:26
S… Speaker 2 (Audio_04_02_2026_15_02_47)
so they remain above zero inventory at any given time, right?
50:30
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Yeah.
50:31
S… Speaker 1 (Audio_04_02_2026_15_02_47)
We will, regardless, we take our inventory, spares, et cetera.
50:36
S… Speaker 1 (Audio_04_02_2026_15_02_47)
We get our rework ready by Monday,
50:39
S… Speaker 1 (Audio_04_02_2026_15_02_47)
and then Sienna will have an internal discussion,
50:43
S… Speaker 1 (Audio_04_02_2026_15_02_47)
let us know what support is required, right?
50:46
S… Speaker 1 (Audio_04_02_2026_15_02_47)
And then we can support accordingly.
50:50
S… Speaker 1 (Audio_04_02_2026_15_02_47)
So anything else we need to discuss?
51:00
S… Speaker 1 (Audio_04_02_2026_15_02_47)
We will, I think Saliden team will expect to hear back from you guys, right?
51:06
S… Speaker 1 (Audio_04_02_2026_15_02_47)
to let us know what would be the next stage, next step forward with this.
51:15
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Internally, we are moving forward to remove shiphold and build products
51:21
S… Speaker 1 (Audio_04_02_2026_15_02_47)
based on the new updated final test there.
51:25
S… Speaker 3 (Audio_04_02_2026_15_02_47)
Yeah, I guess it would be good to have another call, Linda, next week
51:30
S… Speaker 3 (Audio_04_02_2026_15_02_47)
to see the updated status of these corrective action and preventive actions,
51:37
S… Speaker 3 (Audio_04_02_2026_15_02_47)
as well as to get an update on your inventory and turnaround time.
51:41
S… Speaker 3 (Audio_04_02_2026_15_02_47)
And then internally, we're going to discuss, like, based on what you would offer
51:47
S… Speaker 3 (Audio_04_02_2026_15_02_47)
for, like, how many we can send you and how many you're going to send us.
51:51
S… Speaker 3 (Audio_04_02_2026_15_02_47)
So that can help us to decide, basically, what do we do with those 700 units.
51:59
S… Speaker 3 (Audio_04_02_2026_15_02_47)
Plus, I think we will have an internal discussion to go about the units that are already shipped.
52:08
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Okay.
52:10
S… Speaker 1 (Audio_04_02_2026_15_02_47)
So when do you think you guys have a discussion first internally,
52:15
S… Speaker 1 (Audio_04_02_2026_15_02_47)
then we set up a meeting jointly to talk about how we support your plan?
52:20
S… Speaker 3 (Audio_04_02_2026_15_02_47)
I guess we need to hear from you, basically, to know how many units you can provide us
52:28
S… Speaker 3 (Audio_04_02_2026_15_02_47)
so we can replace those in our inventory and sending it back to you for rework, right?
52:37
S… Speaker 1 (Audio_04_02_2026_15_02_47)
That affects our plan, right?
52:41
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Yeah, I'm going to reach out and see what we have in terms of material.
52:44
S… Speaker 2 (Audio_04_02_2026_15_02_47)
I think the one thing we just need to validate internally is how many of these
52:47
S… Speaker 2 (Audio_04_02_2026_15_02_47)
we already retested and done up with the new process.
52:54
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Okay.
52:54
S… Speaker 1 (Audio_04_02_2026_15_02_47)
And, yeah, we can provide that information early next week.
53:03
S… Speaker 5 (Audio_04_02_2026_15_02_47)
I'm interested in knowing any cars are inserted so he can engage with the GCC guys.
53:13
S… Speaker 4 (Audio_04_02_2026_15_02_47)
We're not going to be able to find that out, Kevin, right?
53:16
S… Speaker 4 (Audio_04_02_2026_15_02_47)
That's extremely difficult to know, right?
53:18
S… Speaker 4 (Audio_04_02_2026_15_02_47)
You'd have to go get, like, a network inventory export from a given customer
53:22
S… Speaker 4 (Audio_04_02_2026_15_02_47)
to find out what's in there in their shelves.
53:24
S… Speaker 4 (Audio_04_02_2026_15_02_47)
They're going to have to know all the serial numbers.
53:26
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Well, I guess in this case, it's all new deployments, so that makes it a little easier.
53:30
S… Speaker 4 (Audio_04_02_2026_15_02_47)
But, yeah, getting that information out of customers is difficult.
53:35
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Okay.
53:35
S… Speaker 5 (Audio_04_02_2026_15_02_47)
So, I guess we'll have an internal discussion on how we're going to approach rescuing, right?
53:41
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Well, yeah, definitely have to do that.
53:42
S… Speaker 4 (Audio_04_02_2026_15_02_47)
And then I think that someone's got to keep track of all the serial numbers of cards
53:47
S… Speaker 4 (Audio_04_02_2026_15_02_47)
where things have been corrected.
53:51
S… Speaker 5 (Audio_04_02_2026_15_02_47)
Yeah, I think Lumentum can provide that, right?
53:54
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Who's going to do that?
53:56
S… Speaker 5 (Audio_04_02_2026_15_02_47)
I think Lumentum will provide a list of serial numbers of all the fixed...
54:01
S… Speaker 3 (Audio_04_02_2026_15_02_47)
Oh, no, no, about the ones in the field, I guess.
54:05
S… Speaker 3 (Audio_04_02_2026_15_02_47)
William, you mean the ones that are already going to be fixed in the field or Lumentum ones?
54:13
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Well, again, there's different buckets, right?
54:15
S… Speaker 4 (Audio_04_02_2026_15_02_47)
So, there's a bucket of the 700 that haven't been shipped yet,
54:18
S… Speaker 4 (Audio_04_02_2026_15_02_47)
and then you guys can figure out how to deal with that
54:20
S… Speaker 4 (Audio_04_02_2026_15_02_47)
and keep track of which serial numbers have or have not been fixed so we know.
54:25
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Yeah, we do.
54:25
S… Speaker 4 (Audio_04_02_2026_15_02_47)
And then there's a separate problem of, okay,
54:27
S… Speaker 4 (Audio_04_02_2026_15_02_47)
how are we going to manage and keep track of what's been done in the field
54:31
S… Speaker 4 (Audio_04_02_2026_15_02_47)
to know which cards and things have been adjusted?
54:34
S… Speaker 4 (Audio_04_02_2026_15_02_47)
And that will have to be an internal discussion on our end
54:38
S… Speaker 4 (Audio_04_02_2026_15_02_47)
how to coordinate that with the tech support team.
54:40
S… Speaker 1 (Audio_04_02_2026_15_02_47)
So, normally, for cards sent back to Lumentum is going through RMA process.
54:45
S… Speaker 1 (Audio_04_02_2026_15_02_47)
We have the whole database, and that's all tracked.
54:49
S… Speaker 1 (Audio_04_02_2026_15_02_47)
And Ciena also has DMS track all those.
54:52
S… Speaker 1 (Audio_04_02_2026_15_02_47)
So, as long as come back through RMA process, it's all tracked,
54:56
S… Speaker 1 (Audio_04_02_2026_15_02_47)
both on Ciena's side and our side.
54:59
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Sure.
55:00
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Great, as long as there's an easy way to identify which cards have been reprocessed at Momentum, so we know, yeah, at this point in time, here's a list of all the serial numbers of cards we know that are good, that'll be helpful.
55:12
S… Speaker 4 (Audio_04_02_2026_15_02_47)
But that's just one small piece of the overall challenge, given how many cards have been shipped so far to date.
55:19
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Yeah, we can track that, if it's coming back to us, it's through our May.
55:23
S… Speaker 7 (Audio_04_02_2026_15_02_47)
Okay, so who's going to drive the internal discussion, Willem, or the, we're going to approach the ones that are not replacing hardware?
55:41
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Yeah, I think we'll have to have a separate discussion, I mean, my knee-jerk reaction is, you know, this is a card manufacturing issue, and it should be someone from the hardware quality team being the quarterback here to coordinate all of this, but I'm assuming some people on the call might have a different opinion on who that should be, and I think we'll have to take that offline,
56:01
S… Speaker 4 (Audio_04_02_2026_15_02_47)
because this is going to be a major activity and a major project.
56:19
S… Speaker 7 (Audio_04_02_2026_15_02_47)
Well, I'll tear myself out, right? I don't have the visibility on what's out there, so, but I'd definitely put this out there.
56:26
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Yeah, no, I, you know, from a software development team perspective, I don't think we're going to be driving this, right?
56:32
S… Speaker 4 (Audio_04_02_2026_15_02_47)
This has got to be a customer-facing, and I would think it would fall under the scope of the hardware quality team to manage and project manage this.
56:40
S… Speaker 4 (Audio_04_02_2026_15_02_47)
I don't know if Greg or Saeed, you guys have a view, but I think we'll have to figure that out internally, right?
56:46
S… Speaker 4 (Audio_04_02_2026_15_02_47)
It's like, who's going to run with this?
56:48
S… Speaker 6 (Audio_04_02_2026_15_02_47)
Yeah, I mean, we'll have the list that's going to be bad.
56:51
S… Speaker 6 (Audio_04_02_2026_15_02_47)
Saeed will open up an SPI, which has the entire list.
56:54
S… Speaker 6 (Audio_04_02_2026_15_02_47)
We can say that any time a DMS ticket is closed after that, those are cleared.
57:00
S… Speaker 6 (Audio_04_02_2026_15_02_47)
Somebody from GCC is going to have to tell us units that they clear from the field,
57:03
S… Speaker 6 (Audio_04_02_2026_15_02_47)
otherwise they'll just remain as if they weren't fixed until they ever came back, so there's going to be a lot of that.
57:10
S… Speaker 4 (Audio_04_02_2026_15_02_47)
Yeah, and then there's the whole proactive measure.
57:12
S… Speaker 4 (Audio_04_02_2026_15_02_47)
It's like, how much are we going to do proactively on cards that are potentially in systems that are running a release prior to 1696 to do that,
57:21
S… Speaker 4 (Audio_04_02_2026_15_02_47)
or are we going to wait until they go to upgrade and deal with them when problems get reported?
57:26
S… Speaker 4 (Audio_04_02_2026_15_02_47)
I would think we'd want to be proactive and prevent problems before they emerge.
57:31
S… Speaker 5 (Audio_04_02_2026_15_02_47)
I agree with that.
57:32
S… Speaker 4 (Audio_04_02_2026_15_02_47)
We've got to sort out that whole strategy and who's going to coordinate and do all of that.
57:36
S… Speaker 3 (Audio_04_02_2026_15_02_47)
Yeah, I agree with that.
57:37
S… Speaker 6 (Audio_04_02_2026_15_02_47)
If GCC agrees to do the pre-upgrade check, then I mean, I don't want to say it lightly,
57:42
S… Speaker 6 (Audio_04_02_2026_15_02_47)
but I care less about what's in the field today, and I'd rather tackle the other two areas that we've been talking about,
57:47
S… Speaker 6 (Audio_04_02_2026_15_02_47)
but that remains to see if they'll sign up for that based off of resourcing.
57:52
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Yeah.
57:56
S… Speaker 6 (Audio_04_02_2026_15_02_47)
So, Saeed, I guess we need an internal call, obviously, to hash out the rest of that,
58:00
S… Speaker 6 (Audio_04_02_2026_15_02_47)
and then in parallel, Lamentum can get back to us on what they have to help us start the swap process.
58:07
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Sounds good.
58:10
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Okay.
58:12
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Okay.
58:13
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Yeah, four o'clock, another Siena meeting.
58:16
S… Speaker 3 (Audio_04_02_2026_15_02_47)
So, it's all good?
58:18
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Yeah, just a few action items I know get us and out.
58:22
S… Speaker 2 (Audio_04_02_2026_15_02_47)
Thank you.
58:24
S… Speaker 7 (Audio_04_02_2026_15_02_47)
Thank you.
58:25
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Sounds good.
58:25
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Thank you.
58:26
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Have a good day.
58:27
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Yeah.
58:28
S… Speaker 1 (Audio_04_02_2026_15_02_47)
Bye-bye.
58:29
S… Speaker 8 (Audio_04_02_2026_15_02_47)
Thanks, everyone.

हे transcript AI (स्वयंचलित वक्तृत्व ओळख) द्वारे बनविले गेले आहे. त्रुटी असू शकते - गंभीर वापर करीता मूळ ऑडिओशी तुलना करा. AI करार

❤️ ला प्रेम करा STT.ai? आपल्या मित्रांना सांगा!
सारांश
या कोशाच्या संपादनासाठी त्यांनी एक स्वतंत्र कोशकार मंडळ स्थापन केले.
सारांश करीता...
या प्रतविषयी AI ला विचारा
या योजनेत काही गोष्टींचा समावेश असतो - योजनेत सहभागी होण्यासाठी आवश्यक कागदपत्रे व माहिती.
दुसरे प्रतलेखन करा