Consume XML from a WEB service

**pablokuster@gmail.com** · 2012-10-12, 09:44

Hello community, actually I'am trying to port a very sample class library (see attachment), developed and tested in VFP9, used to query an XML database (marklogic 6.0) using RESTfull protocol to formulate the request as (http://localhost:8011/v1/search?q=bo...&pageLength=30).

In VFP I use the MSXML.XMLHTTP60 activeX object to open and send the request to the Marklogic rest server installed on my PC and all work fine, from the request to the load of the XML response on a local cursor.

I try to use this class in lianja but the method open to prepare the request not work, always return .F. and not a number > 0 that indicate that the port was open without error. Follow the class code:

Code:

************************************************************************
DEFINE CLASS ml_server
************************************************************************
oHT			= .null.	// Reference msxml http interface.
server 		= 'http://127.0.0.1'  // rest server IP (localhost).
port 		        = 8011	// Rest server http port.
rest		        = .null.	// http rest request URL.
page  		= 1		// Current paging page.
pageLength        = 10		// Number or results per page
user		        = ''		// Request user.
pssw 		        = ''		// Request password.
results		= 0	        // Results count of last query.
qy_time		= 0		// Query cost in seconds.
*-----------------------------------------------------------------------
FUNCTION init()
*-----------------------------------------------------------------------
** Build HTTP url prefix for REST query
this.rest = 'http://' + this.server + ':' + TOSTRING(this.port) + '/v1/'

** Default user and password
this.user = 'admin'
this.pssw = 'admin'
** Create reference to msxml http interface.
this.oHT = CREATEOBJECT("MSXML2.XMLHTTP.6.0")
ENDFUNC
*-----------------------------------------------------------------------
FUNCTION sh_search(cQuery as String,nStart as Integer,nPageLength as Integer)
*-----------------------------------------------------------------------
LOCAL oHT as MSXML2.XMLHTTP60
LOCAL cUrl as String,nOk as Integer

m.oHT  = this.oHT
this.pageLength = IIF(VARTYPE(m.nPageLength) == "N",m.nPageLength,this.pageLength)
this.start = IIF(VARTYPE(m.nStart) == "N",m.nStart,1)

** Build rest query url and send query to server.
m.cUrl = this.rest + 'search?q=' + m.cQuery + '&start=' + TOSTRING(this.start) + '&pageLength=' + TOSTRING(this.pageLength) + '&format=xml'
m.nOK  = m.oHT.open("GET",m.cUrl,.T.,this.user,this.pssw)

WAIT TOSTRING(m.nOK) window

** Execute query.
m.nOK = this.sh_execute(m.oHT)

DO CASE
CASE m.nOk = 0	&& Query success.
	WAIT 'Query success, collectin results into cursor' WINDOW NOWAIT
	this.sh_XmlToCursor(m.oHT)
	m.oHT = .null.
	SELECT xQuery
	GO TOP IN xQuery
	BROWSE LAST
CASE m.nOk = 10	&& Query timeout.
	MESSAGEBOX('Query timeout')
CASE m.nOK = -1
	wait 'No results found' window nowait	
ENDCASE
*-----------------------------------------------------------------------
FUNCTION sh_execute(m.oHT as MSXML2.XMLHTTP60)
*-----------------------------------------------------------------------
LOCAL tStart as Integer

m.tStart = SECONDS()
m.oHT.send()
DO WHILE .T.
	WAIT "Waiting for query response" WINDOW NOWAIT
	DO CASE
	CASE m.oHT.readyState = 4
		wait "Results found" window nowait
		RETURN 0
	CASE SECONDS() - m.tStart > 10
		wait "Request time out" window nowait
		RETURN 10
	OTHERWISE
		RETURN -1
	ENDCASE
ENDDO
ENDFUNC
*-----------------------------------------------------------------------
FUNCTION sh_XmlToCursor(m.oHT as MSXML2.XMLHTTP60)
*-----------------------------------------------------------------------
LOCAL oDoc as MSXML2.DOMDocument60
LOCAL oRps as MSXML2.IXMLDOMNodeList,oRsp as MSXML2.IXMLDOMElement
LOCAL K as Integer,I as Integer

** Create xQuery cursor to hold response results.
IF USED("xquery")
	WAIT 'Close table ml_search' WINDOW
	USE IN xquery
ENDIF
USE ml_search IN 0 ALIAS xquery
WAIT left(m.oHT.responseXML.xml,30) WINDOW

** Load response into DOM xml document.
m.oDoc = CREATEOBJECT("MSXML2.DOMDocument.6.0")
m.oDoc.setProperty("SelectionNamespaces","xmlns:search='http://marklogic.com/appservices/search'")
m.oDoc.loadXML(m.oHT.responseXML.xml)

** Create required data model objects.
m.oShRsp = CREATEOBJECT('empty')

** Get response elements
m.oRps = m.oDoc.documentElement.selectNodes('/search:response')

FOR m.I = 0 TO m.oRps.length - 1
	m.oRsp = m.oRps.item(m.I)
	= ADDPROPERTY(m.oShRsp,'total',INT(VAL(m.oRsp.getAttribute('total'))))
	= ADDPROPERTY(m.oShRsp,'start',INT(VAL(m.oRsp.getAttribute('start'))))
	= ADDPROPERTY(m.oShRsp,'pgLen',INT(VAL(m.oRsp.getAttribute('page-length'))))

	INSERT INTO xQuery (index,match) VALUES (-1,'Results = ' + TRANSFORM(m.oShRsp.total))
** Get serch results element node list list.
	LOCAL oRts as MSXML2.IXMLDOMNodeList,oRst as MSXML2.IXMLDOMElement
	
** Create required data model objects.
	m.oShRst = CREATEOBJECT('empty')
	m.oRts = m.oRsp.selectNodes('./search:result')
	FOR EACH m.oRst IN m.oRts
		= ADDPROPERTY(m.oShRst,'index',INT(VAL(m.oRst.getAttribute('index'))))
		m.K = AT(':/',m.oRst.getAttribute('uri'))
		= ADDPROPERTY(m.oShRst,'uri',SUBSTR(m.oRst.getAttribute('uri'),m.K-1))
		= ADDPROPERTY(m.oShRst,'docName',JUSTSTEM(m.oShRst.uri))
		= ADDPROPERTY(m.oShRst,'score',INT(VAL(m.oRst.getAttribute('score'))))
		= ADDPROPERTY(m.oShRst,'confidence',VAL(m.oRst.getAttribute('confidence')))
		= ADDPROPERTY(m.oShRst,'fitness',INT(VAL(m.oRst.getAttribute('fitness'))))
		
	** From current result, get search match
		LOCAL oMhs as MSXML2.IXMLDOMNodeList,oMth as MSXML2.IXMLDOMElement
		
		m.oMhs = m.oRst.selectNodes('./search:snippet/search:match')
		FOR EACH m.oMth IN m.oMhs
			= ADDPROPERTY(m.oShRst,'match',m.oMth.text)
			= ADDPROPERTY(m.oShRst,'navpath',m.oMth.getAttribute('path'))
			
		** Add current search result to cursor xquery.
			INSERT INTO xquery FROM NAME m.oShRst
		ENDFOR
	ENDFOR
ENDFOR
ENDFUNC
ENDDEFINE

I'am looking to do the same using Lianja, without the MSXML ActiveX object, but not found any class that have this king of services.

**lianjasupport** · 2012-10-12, 09:48

If you SET DEBUG ON in the command console and then run it, the debug.txt file in the lianja\beta directory should let you see what the method signatures are and what it causing the issue.

i agree it would be nice to have an xmlparser class exposed as standard that is cross platform and does not use activex which is windows specific.

**lianjasupport** · 2012-10-12, 09:51

Also if you

SET DEBUGOUT ON

and put

DEBUGOUT "some message"

these will be included in the debug,txt file also which should help to track the problem down.

**lianjasupport** · 2012-10-12, 09:54

Ah... Looking at your code further you can't use foreach on activex collections like that in Lianja you must get each item by index e.g item( n ).

**pablokuster@gmail.com** · 2012-10-13, 02:57

Originally Posted by lianjasupport

i agree it would be nice to have an xmlparser class exposed as standard that is cross platform and does not use activex which is windows specific.

Yes we think the same, implement a multi platform XML PE, may be, is not the best solution, but what about some integration that can be less difficult. We ask for solutions like XML adapter that load XML without any kind of validation, because the server that receives the request and produce the response make this work in the model of web services REST or SOAP, and load this data in one or more specified tables.

This solution require only a class with the functionality of make a http request to a server and the necessary events and methods to control when the response arrive (may be with asynchronous request).

Code:

<invoice date="" number="">
   <customer country="" language="">
       <id>nnn</id>
       <name>
           <first>aaaaa</first>
           <middle>bbbb</middle>
           <last>cccccc</last>
       </name>
       <invoiceAddress></invoiceAddress>
       <shipAdress></shipAdress>
       <!-- other data -->>
    </customer>
    <detail>
        <item="" desc="" quantity="" price="" ......>
        <item="" desc="" quantity="" price="" ......>
        <item="" desc="" quantity="" price="" ......>
     <detail>
</invoice>

After this, became the XML adapter that describe how to load data in each table, es: customer and invoice, that define the schema of data to load and the binding of each column against corresponding data into XML document using xpath, relations that can be 1:1 (customer data) or 1:n (invoice details)

We continued to try with msxml to go through this paradigm, but we propose to lianja developer team to consider the possibility of integrate a solution of this kind, in today days very useful.

TIA.

**barrymavin** · 2012-10-13, 03:02

Ok then submit an enhancement request on lianja.com. We use an XML parser internally maybe we can expose it as a standard system class.

**barrymavin** · 2012-10-13, 03:05

I'd also like to point out that we do in fact support loading tables from XML in our SQL but the format of that XML has to be in Microsoft ADO format. There are also built-in functions that will map XML to and from an object.

**pablokuster@gmail.com** · 2012-10-13, 03:49

Thank very much barry we write the request.

We try your useful suggestions about the debug output activation and the debug.txt ouput:

Error reported from file lib_exp1.c line 499 n=42
cmdline=m.nPageLength
Error reported from file lib_exp1.c line 499 n=42
cmdline=m.

nPageLenght and nStart are parameters passed to the ml_search method, but no have idea what is happen, can you give us some help

Tia.

**barrymavin** · 2012-10-13, 04:01

have a look in debug_client.txt

**pablokuster@gmail.com** · 2012-10-13, 05:03

Follow the relevant content of debug_client.txt

Class Activex Object OHT(145406800,249964368) Method: name='open'
File ..\..\rco_activex.cpp at line 620 method=open(QString,QString,QVariant,QVariant,QVar iant) args=5
File ..\..\dao.cpp at line 948 unknown type=0
File ..\..\rco_activex.cpp at line 626 type=U v=
Class Activex Object MSXML2.XMLHTTP.6.0(145410640,249964368) Method: name='send'
File ..\..\rco_activex.cpp at line 620 method=send() args=0
File ..\..\dao.cpp at line 948 unknown type=0
File ..\..\rco_activex.cpp at line 626 type=U v=
Class Activex Object OHT(145406800,249964368) GetProperty: name='readyState'
File ..\..\rco_activex.cpp at line 721 getProperty() name=readystate value=1 type=2 typename=int
File ..\..\rco_activex.cpp at line 726 getProperty() name=readystate value=1 type=2 typename=int value.type=N

and the debug.txt:

Error reported from file lib_exp1.c line 499 n=42
cmdline=m.nPageLength
Error reported from file lib_exp1.c line 499 n=42
cmdline=m.nStart

the tho variables (m.nPageLength,m.nStart) are parameters received in the method sh_search of my class:

FUNCTION sh_search(cQuery as String,nStart as Integer,nPageLength as Integer)

and not have nothing to do with the open or send methods of MSXML2.XMLHTTP class.

looking at debug_client.txt the only relevant content that I see are relative previous open and send methods.

tia for your help.

note: let me know if you need all the content of client_debug.txt