GCEV_TASKFAIL raised spuriously

[Guest miva]

Hi,

 

Since upgrading to the latest version of VG, we seem to be having some issues with our recording modules which we use for manual voicemail.

 

We manually set the Silence detect levels to 0 then send the call into a recording module, however we finding the system is frequently hanging up on callers before the recording begins and log suggests it may be finding silence.

 

Attached is an example where it hung up on the caller and did not let them leave a message.

 

This problem is common across all our servers which are running v7.3.4391.19239

 

Can you please let us know the solution.

 

Regards,

Ben.

RecordingIssue.txt

[SupportTeam]

Could you please post the ktTel trace excerpt capturing this call. We can then see why the Dialogic card completed recording of file so quickly.

[Guest miva]

Please find kttel attached.

 

Note, the drive and folders do exist and the file "VM_120523_100045921.wav" was created as a zero length file.

 

Regards,

Ben.

Recording_kttel.txt

[SupportTeam]

Please update your system to this version of VoiceGuide:

[link removed]

It looks like the Dialogic drivers are responding in unusual way to commands related to record start, but this new version (above) will accept these unusual responses.

Which Dialogic Service Update version is installed on this system?

How often was the record issue occurring? Every time a record was attempted, or on a percentage of calls?

Can you post the full ktTel trace file (.ZIPed) ?

[Guest miva]

The record issue has been happening on a percentage of calls, perhaps 20 or 30% of the time. We also use another recording module to record two line conferenced calls does not appear to exhibit the same problem.

 

Help about from DCM shows Dialogic System Release 6.0 PCI build 235 Production.

 

Please find full kttel zip attached.

0523_ktTel.zip

[SupportTeam]

Please update the Dialogic drivers to the version that is downloadable from the VoiceGuide Downloads page.

The drivers that are downloadable from the VoiceGuide Downloads page are the ones that are intended for use with current version of VoiceGuide.

 

If you still encounter problems when using the "120524" VoiceGuide build provided on system using the Dialogic drivers from the VoiceGuide Downloads page then please post ktTel traces as before.

[Guest miva]

We still seem to be having this issue intermittantly.

 

Extract of log attached from vgEngine and ktTel.

LogExtract.txt

[SupportTeam]

Can you post a longer ktTel trace? long enough to include the previous call on that same port.

 

Also please post the first 20 or lines that are written to ktTel trace at system restart time - that show software versions etc.

 

Please .ZIP up traces before posting them.

[SupportTeam]

Please also .ZIP up and post the Dialogic RTF logs cover this time. Those logs should provide more information why Dialogic issued a "TASKFAIL" event.

 

094434.140 3420 13 ev GCEV_TASKFAIL crn=280005b

[Guest miva]

Please find updated logs attached as requested. vgEngine log now includes snippet of start with S/W version information as does ktTel log.

Logs-9th-Aug-2012.zip

[SupportTeam]

Please .ZIP up and post the ktTel log from 9:20 till 9:50

 

The headers included seem to be from ktTts, not ktTel. Please include ktTel start header.

[Guest miva]

Sorry, correct log extract including header attached this time including from 0759 when the service was reset.

Log-ktTel-9th-Aug-2012.zip

[SupportTeam]

There seems to be some issues on Dialogic level that give rise to the GCEV_TASKFAIL being issued by Dialogic.

 

Will look into it and advise.

 

08/09/2012 09:32:55.671   1728        3428 libisdnr..ll            ERR1                   ::::> cc_GetCRN() returns 0x386 due to (invalid ec_crn)
08/09/2012 09:32:55.671   1728        3428 spwrgcis                ERR1         gcis                  gcis_GetCRN() returns -1 due to cc_GetCRN() failed

08/09/2012 09:44:34.156   1728        3428 libisdnr..ll            ERR1                   ::::> cc_GetCRN() returns 0x386 due to (invalid ec_crn)
08/09/2012 09:44:34.156   1728        3428 spwrgcis                ERR1         gcis                  gcis_GetCRN() returns -1 due to cc_GetCRN() failed

093255.640  3420  10 ev    GCEV_TASKFAIL crn=2800048

094434.140  3420  13 ev    GCEV_TASKFAIL crn=280005b

[Guest miva]

Thanks, your assistance is appreciated.

[SupportTeam]

The version below ignores any GCEV_TASKFAIL being issued by Dialogic while the two-line record is underway:

[old link deleted]

Please update system to this version and see if this workaround is sufficient.

The GCEV_TASKFAIL events look like a bug in Dialogic drivers, so perhaps upgrading to newer Dialogic drivers when they become available will fix this bug.

[SupportTeam]

Also to have the Dialogic RTF logs to store more logging information it is a good idea to edit the Dialogic RtfConfigWin.xml file (in Dialogic's cfg directory) to enable more logging.

In RtfConfigWin.xml set :

 

state="1"

 

for all entries in these sections:

 

spwrgcis

libisdnr4

isdnspan

gc (LIBGC)

 

 

and set the Logfile section of header to:

 

 

<Logfile duplicate_to_debug_console="0" log_format="text" maxbackups="100" path="$(INTEL_DIALOGIC_DIR)\log" preserve_maxbackups="100" preserve_size="10000" size="10000"/>

 

 

 

When we have the Dialogic RTF traces along with VoiceGuide traces we can then fully see what is happening on the system.

[Guest miva]

Just saw a line clear down without going through notify script. Upon review of log files found another instance of GCEV_TASKFAIL. Logs from engine and tel attached.

 

Will update dialogic logging now so we can hopefully capture any future instances and provide more information. Will the service need to be restarted?

Logs-15-8.zip

[SupportTeam]

Could you please upload the Dialogic RTF logs that cover this time?

[SupportTeam]

Have the posted ktTel and vgEngine traces had any parts of traces removed? (apart from the section between the header and the start of trace, and after the 120354.093 timestamp)

[Guest miva]

Nothing was removed within that time period, we simply cut it out to exclude before and after.

 

Here is dialogic log from same period. It's quite short so will post it here:

 

08/15/2012 12:00:36.015 1708 2244 isdnspan ERR1 isdnrecv dtiB1T1 ====> CancelSingleRequest: Failed with SRAM Error Code 0xe000000d

08/15/2012 12:00:36.015 1708 2244 libisdnr..ll dtiB1T1 ERR1 dtiB1T1 ----> IsdnCancelSingleRequest(LineDev:2) returns: 0x5d8 due to (SrlSendAtomicRequestAndWait failed)

08/15/2012 12:02:58.359 1708 2244 gc ERR1 gclib ----- GC_LDVERIFY macro - functbl->ccfp_GetXmitSlot is NULL

08/15/2012 12:02:58.375 1708 2244 gc ERR1 gclib ----- GC_LDVERIFY macro - functbl->ccfp_GetXmitSlot is NULL

08/15/2012 12:03:09.093 1708 2592 libisdnr..ll dtiB1T1 ERR1 dtiB1T1 ::::> cc_GetInfoElem() returns: 0x303 due to (GetIE[0].length == 0)

08/15/2012 12:03:09.093 1708 2592 spwrgcis ERR1 gcis dtiB1T1 ::::> gcis_GetInfoElem() returns -1 due to cc_GetInfoElem(0x0) failed

08/15/2012 12:03:09.093 1708 2592 gc ERR1 gclib dtiB1T1 ::::> gc_GetInfoElem(linedev:2) - returns:-1

08/15/2012 12:03:09.093 1708 2592 libisdnr..ll dtiB1T1 ERR1 dtiB1T1 ::::> cc_Internal_GetCallInfo(Crn:0x80000001) failed due to (Unknown nInfoID=0x102)

08/15/2012 12:03:09.093 1708 2592 spwrgcis ERR1 gcis gcis_GetCallInfo() returns -1 due to cc_GetCallInfo(0x80000001) failed

08/15/2012 12:03:09.093 1708 2592 gc ERR1 gclib ::::> gc_GetCallInfo(crn:0x2800001h, info_id:258) - returns;-1

08/15/2012 12:03:09.140 1708 2592 gc ERR1 gclib ----- GC_LDVERIFY macro - functbl->ccfp_GetXmitSlot is NULL

08/15/2012 12:03:19.453 1708 3140 libisdnr..ll ERR1 ::::> cc_GetCRN() returns 0x386 due to (invalid ec_crn)

08/15/2012 12:03:19.453 1708 3140 spwrgcis ERR1 gcis gcis_GetCRN() returns -1 due to cc_GetCRN() failed

[SupportTeam]

Will update dialogic logging now so we can hopefully capture any future instances and provide more information. Will the service need to be restarted?

Yes. Dialogic service will need to be restarted.

[SupportTeam]

Could you please collect system snapshot by executing:

C:\Program Files\Dialogic\bin\its_sysinfo.exe

 

and then .ZIP up its output and post it here.

[Guest miva]

Please find ZIP attached.

its_sysinfo.zip

[SupportTeam]

Dialogic have asked to enable fuller RTF tracing on this machine, as well as to perform ISDN level tracing.

 

ISDN tracing is to check whether an unexpected ISDN message from switch gives rise to the GCEV_TASKFAIL event being raised.

 

Could you please .ZIP up and post the RtfConfigWin.xml file from your system?

[SupportTeam]

Setting up ISDN trace - instructions:

 

How can the ISDIAG utility be used to obtain a D-Channel trace?

 

Reason for the issue:

D-Channel traces can be useful for troubleshooting issues involving ISDN calls.

 

Fix / Solution:

A 'D-Channel’ trace originates as a binary reading of the information passed at the ISDN layer 2 level. It will capture all messages delivered to and from the Dialogic® board. The following steps show how to obtain this trace, and how to convert it into a readable format.

 

When using the ISDIAG utility to obtain a trace while a separate program is making/receiving ISDN calls, use the "r" (trace mode) flag when initiating ISDIAG. Using this flag will stop ISDIAG from issuing a cc_WaitCall() on the specified channel, leaving it available for another application to use the channel.

 

LOCATION: The ISDIAG tool is located in the Dialogic\Bin directory.

 

USAGE:

 

isdiag <board> <channel> <boardtype> <type> <voice>

 

 

board =====> board #

channel ===> 1-23 for T1

===> 1-30 for E1

===> 1-2 for BRI

boardtype => 't' = T1

=> 'e' = E1

=> 'b' = BRI

=> 'b2' = BRI/2VFD

type =====> 'p' = PEB // No longer supported

's' = SCBUS or CTBUS [Only use 's' for "type"]

trace mode => 'r' = Trace mode, No waitcall issued at startup

default: Normal mode, waitcall issued at startup

voice =====> 'v' = voice supported (span cards)

default: no voice support (dti cards)

 

Examples:

 

isdiag 1 1 t s

 

 

(this will run isdiag in user mode, and will be ready to make or receive calls)

 

isdiag 1 1 t s r

 

 

(this will run isdiag in 'r' resource mode, and will not be usable for making/receiving calls -- use this mode to capture a D-Channel trace. The user will run an application for call control. If the user does not have a program for call control, then the user should not use the 'r' resource mode switch).

 

 

To start a trace:

 

From the ISDIAG menu screen, do the following:

 

1) Select option 10 to start/stop/browse trace

 

2) Select option 1 to start a trace

 

3) You will be prompted for a file name. Use a file name that you will remember in case you need it later, and enter a full pathname (e.g. c:\program files\dialogic\bin\trace1.log).

 

4) After entering a file name, you will be taken back to the original menu screen.

 

5) When the test is complete, select option 10 to start/stop/browse trace, and then select option 2 to stop the trace.

 

 

To read a trace:

 

After the trace has been stopped, the .log file that’s generated is an unreadable binary file and it must be parsed into a readable format. ISDIAG can do this automatically, or it is possible to have an outside program parse the information.

 

To read a trace using ISDIAG:

 

1) Select option 10 from the main menu

 

2) Select option 2 to Stop the trace – you will be taken back to the main menu.

 

3) Select option 10 again from the main menu.

 

4) Select option 3 to browse the trace.

 

To read a trace outside ISDIAG:

 

1) Run a utility that can parse ISDN layer 2 data on the .log file (i.e. isdtrace or pritrace, at least one of which is included in any Dialogic release which has ISDIAG) -- to run pritrace on a file in the same directory as it, the following command would be entered:

 

pritrace filename.log [destination filename]

 

 

If no destination filename is entered, the same name will be used, with extension .res.

 

2) The .res file will be readable with any text editor

[Guest miva]

Please find RtfConfigWin.xml ZIP attached.

RtfConfigWin.zip

[SupportTeam]

Had response from Dialogic about this.

 

Dialogic advice is that there are some issues with the Dialogic drivers installed on this machine. Dialogic advises that the drivers currently present are not a clean install, and DLLs from older driver version are present on system.

 

Their recommendation is to uninstall the current drivers, then ensure everything has been fully removed by running the dlgc_rel_clean.bat file (in Dialoigc's \bin\ directory - you would need ot make a separate copy of this file before uninstall).

 

Then install new drivers (you can download these from our WWW Downloads page).

 

After new drivers are installed Dialogic asks to run the ISDN trace to check whether any ISDN messages arrive at the time when the TASKFAIL event is raised.

 

Would recommend to first perform a clean driver install as outlined above, and then if TASKFAIL event is still raised by the drivers then run the ISDN trace.

 

Also, if these events still arise after driver install please advise how often they arise.