Dropped Calls

[mwattles]

Voiceguide,

I have recently taken over responsibility of our IVR Systems . We have started having a problem with the systems dropping a large number of calls when it tries to transfer the calls to our avaya system. This is happening on both of our IVRs that are running basically the same scripts. We are using dialogic cards software version 6.0 and we are running voiceguide version 6.0.3235 the IVR is trying to transfer to an Avaya G3r V9 Each Trunk Group has 92 circuits with 2 trunk groups per IVR. The circuit cards are TN464G series DS1 cards and the circuits are built as ISDN.

 

When the system starts failing on the transfers to the Avaya we are having to reboot both IVRs and that provides a temporary fix to the problem. At this point we are having to reboot the systems multiple times throughout the day. We are trying to determine if the problem is on the dialogic, voiceguide, router, switch, t1, or avaya. Have you seen this type of issue before? What troubleshooting steps would you suggest?

 

Thanks,

 

[SupportTeam]

The quickest way to fix an issue that wasn’t there before is to figure out what has changed with the system. The problems is more then likely caused by whatever changes were made to this previously working system.

 

Do you perhaps know the time/date of when this started occurring?

 

Did this start occurring on both system simultaneously?

 

Can you recall if anything was changed with the T1 lines connected to these servers at around the time when this problem started occurring?

 

 

Could you please post a copy of VoiceGuide's Trace Logs which captures the problem.

 

Enable logging by setting the log levels to 10 in VG.INI as per below:

[Log]

 

VoiceGuide=10

ktTel=10

Then restart VG and wait until the problems start occurring.

 

Trace files will be created in VG's \log\ subdirectory.

 

Please post the traces and the VoiceGuide script used.

 

When posting traces/scripts please .ZIP them up and post them as attachments.

 

When posting the traces please indicate the time at which the problems started occurring and provide information which will let us find at least one of the calls within the trace file. Eg. CallerID and time of call would be sufficient.

 

If the .ZIPed trace files are too big to post then please email them to support@voiceguide.com and in email include a link to this forum thread.

 

 

We can see from the traces what reason is given for the calls being terminated.

[mwattles]

No changes have been made to the systems in the last 4 months. The problems started about a month ago.

 

I have attached a few documents to this post. If you look at the spreadsheet named TG4 IVR1 to avaya.xls you will see that the t1 2e09 that goes out from the IVR to the Avaya system is not being utilized and the t1 2e13 is only getting partial use. The same is showing on the TG11IVR to Avaya.xls. On this one 3e06 is not being utilized and 3e11 is only getting partial utilization. I have also attached a copy of the log file from the IVR. The number that dropped is coming from 4072154700. Below is the current mapping of the Avaya and IVR System. What piece would be causing the system not to utilize the T1’s?

 

 

Slot IVR # IVR Board Avaya Board Avaya TG Trunk Direction

 

3 2 B2T2 2A03 3 Outbound to IVR

4 1 B4T2 1D07 2 Outbound to IVR

5 1 B1T2 3A05 4 Inbound from IVR

6 1 B2T2 3A07 2 Outbound to IVR

7 1 B3T2 1C18 2 Outbound to IVR

8 1 B4T1 3E16 2 Outbound to IVR

9 1 B1T1 2E10 4 Inbound from IVR

10 1 B2T1 2E13 4 Inbound from IVR

11 1 B3T1 2E09 2 Outbound to IVR

13 2 B3T2 2A15 3 Outbound to IVR

14 2 B3T1 2E05 3 Members Split

15 2 B1T1 3A08 11 Inbound from IVR

17 2 B4T1 3A16 3 Outbound to IVR

18 2 B4T2 3E06 3 Outbound to IVR

24 2 B1T2 3E12 11 Inbound from IVR

23 2 B2T1 3E11 11 Inbound from IVR

 

logs.zip

[SupportTeam]

Trace shows that when call from 4072154700 was eventually transferred (using Dial-and-Conference / Tromboned transfer) to 9,18009793928 the ISDN message received by the Dialogic card (from Avaya) in response to the outbound call was initially OFFERING and then 3 seconds later a DISCONNECT ISDN message arrived. The cause associated with the Disconnect message was "GCRV_BUSY : Line is busy,Destination Busy".

 

The DISCONNECT message arrived form the system to which the T1 on which the outgoing call was placed connects to, so looks like you will now need to determine why the remote system (Avaya) is responding with the DISCONNECT message.

 

Extracted Trace of the call from 4072154700 attached. The OFFERING and DISCONNECT messages can be seen towards the end of the attached extract.

4072154700.txt

[SupportTeam]

Also, as this call is being made on ISDN lines then there is no need to include a comma in the dialed number. So dialing this number would be sufficient:

 

918009793928

 

instead of

 

9,18009793928

 

[mwattles]

We are still seeing a large number of dropped calls on the our 2nd IVR. I have attached a portion of our log file and wanted to know if the line "call progress detection not enabled as ca_intflg == DX_OPTDIS (2)" that is showing in the logfile could be part of our problem. I am also seeing the line "GCRV_CONGESTION" and I was curious if that could be the problem. If you need more of the logfile let me know and I will upload it.

 

Thanks again,

Matt

1211VGM_2.txt

[mwattles]

I don't know if it is needed or not, but this is the entire log file from our testing this morning with our dropped calls.

 

Thanks Again,

Matt

1211vgm.txt

[mwattles]

One final log file. This is the TW logfile from the testing this morning. That is showing the "call progress detection not enabled as ca_intflg == DX_OPTDIS (2)" line

1211tw.zip

[SupportTeam]

We are still seeing a large number of dropped calls on the our 2nd IVR. I have attached a portion of our log file and wanted to know if the line "call progress detection not enabled as ca_intflg == DX_OPTDIS (2)" that is showing in the logfile could be part of our problem. I am also seeing the line "GCRV_CONGESTION" and I was curious if that could be the problem.

 

The trace shows a similar situation as before. The outgoing call leg is disconnected by the remote system. in this trace we can see that the cause for disconnection given is "Congestion". Looks like somewhere along the outgoing call path on of the nodes/switches does not have enough channels available to accommodate the call and hence it drops the call, and indicates "Congestion" as the reason for the call drop.

 

093731.093 007 ocxfn LineMakeCall(linedev=7, hcall=0, 8908, callprog=CONNECT_IMMEDIATELY, timeout=60, params:[0,0,cid=8636780119,<calltype>xt_dc_blind</calltype>]) chdev=8, dtidev=6

...

093731.093 007 gc gc_MakeCall(7, [8908], 60) ok

...

093735.109 007 ev GCEV_DISCONNECTED

093735.109 007 gc GCST_DISCONNECTED status gc:146:[Function is not supported in the current state], cc_err=32777 cclibid=6

093735.109 007 ResultInfo gcValue=1351(0x547) gcMsg=[Congestion on the line] ccLibId=6 ccLibName=[GC_DM3CC_LIB] ccValue=[0x22a|GCRV_CONGESTION|congestion] ccMsg=[Network congestion] additionalinfo=

 

 

 

 

[SupportTeam]

One final log file. This is the TW logfile from the testing this morning. That is showing the "call progress detection not enabled as ca_intflg == DX_OPTDIS (2)" line

 

Attached trace shows that all of the outgoing calls made from this system (around 150 calls are shown in the trace) are dropped immediately after dialing, with "Congestion" given by the remote system as the cause of the call drop.

 

Searching for "ocxfn LineMakeCall" will find the log entries made when the outgoing call is started.

Searching for GCRV_CONGESTION will find the disconnection log entry.

 

The "call progress detection not enabled as ca_intflg == DX_OPTDIS (2)" line does not affect anything here. This log entry just informs us that after call is answered extra call progress is not required.

 

The outgoing call is never even connected/answered by the remote system.

[mwattles]

Thank you for the quick response. I will pass this on to our Avaya Administrator.