PDA

View Full Version : Consume XML from a WEB service


pablokuster@gmail.com
2012-10-12, 09:44
Hello community, actually I'am trying to port a very sample class library (see attachment), developed and tested in VFP9, used to query an XML database (marklogic 6.0) using RESTfull protocol to formulate the request as (http://localhost:8011/v1/search?q=book&start=1&pageLength=30).

In VFP I use the MSXML.XMLHTTP60 activeX object to open and send the request to the Marklogic rest server installed on my PC and all work fine, from the request to the load of the XML response on a local cursor.

I try to use this class in lianja but the method open to prepare the request not work, always return .F. and not a number > 0 that indicate that the port was open without error. Follow the class code:



************************************************** **********************
DEFINE CLASS ml_server
************************************************** **********************
oHT = .null. // Reference msxml http interface.
server = 'http://127.0.0.1' // rest server IP (localhost).
port = 8011 // Rest server http port.
rest = .null. // http rest request URL.
page = 1 // Current paging page.
pageLength = 10 // Number or results per page
user = '' // Request user.
pssw = '' // Request password.
results = 0 // Results count of last query.
qy_time = 0 // Query cost in seconds.
*-----------------------------------------------------------------------
FUNCTION init()
*-----------------------------------------------------------------------
** Build HTTP url prefix for REST query
this.rest = 'http://' + this.server + ':' + TOSTRING(this.port) + '/v1/'

** Default user and password
this.user = 'admin'
this.pssw = 'admin'
** Create reference to msxml http interface.
this.oHT = CREATEOBJECT("MSXML2.XMLHTTP.6.0")
ENDFUNC
*-----------------------------------------------------------------------
FUNCTION sh_search(cQuery as String,nStart as Integer,nPageLength as Integer)
*-----------------------------------------------------------------------
LOCAL oHT as MSXML2.XMLHTTP60
LOCAL cUrl as String,nOk as Integer

m.oHT = this.oHT
this.pageLength = IIF(VARTYPE(m.nPageLength) == "N",m.nPageLength,this.pageLength)
this.start = IIF(VARTYPE(m.nStart) == "N",m.nStart,1)

** Build rest query url and send query to server.
m.cUrl = this.rest + 'search?q=' + m.cQuery + '&start=' + TOSTRING(this.start) + '&pageLength=' + TOSTRING(this.pageLength) + '&format=xml'
m.nOK = m.oHT.open("GET",m.cUrl,.T.,this.user,this.pssw)

WAIT TOSTRING(m.nOK) window

** Execute query.
m.nOK = this.sh_execute(m.oHT)

DO CASE
CASE m.nOk = 0 && Query success.
WAIT 'Query success, collectin results into cursor' WINDOW NOWAIT
this.sh_XmlToCursor(m.oHT)
m.oHT = .null.
SELECT xQuery
GO TOP IN xQuery
BROWSE LAST
CASE m.nOk = 10 && Query timeout.
MESSAGEBOX('Query timeout')
CASE m.nOK = -1
wait 'No results found' window nowait
ENDCASE
*-----------------------------------------------------------------------
FUNCTION sh_execute(m.oHT as MSXML2.XMLHTTP60)
*-----------------------------------------------------------------------
LOCAL tStart as Integer

m.tStart = SECONDS()
m.oHT.send()
DO WHILE .T.
WAIT "Waiting for query response" WINDOW NOWAIT
DO CASE
CASE m.oHT.readyState = 4
wait "Results found" window nowait
RETURN 0
CASE SECONDS() - m.tStart > 10
wait "Request time out" window nowait
RETURN 10
OTHERWISE
RETURN -1
ENDCASE
ENDDO
ENDFUNC
*-----------------------------------------------------------------------
FUNCTION sh_XmlToCursor(m.oHT as MSXML2.XMLHTTP60)
*-----------------------------------------------------------------------
LOCAL oDoc as MSXML2.DOMDocument60
LOCAL oRps as MSXML2.IXMLDOMNodeList,oRsp as MSXML2.IXMLDOMElement
LOCAL K as Integer,I as Integer

** Create xQuery cursor to hold response results.
IF USED("xquery")
WAIT 'Close table ml_search' WINDOW
USE IN xquery
ENDIF
USE ml_search IN 0 ALIAS xquery
WAIT left(m.oHT.responseXML.xml,30) WINDOW

** Load response into DOM xml document.
m.oDoc = CREATEOBJECT("MSXML2.DOMDocument.6.0")
m.oDoc.setProperty("SelectionNamespaces","xmlns:search='http://marklogic.com/appservices/search'")
m.oDoc.loadXML(m.oHT.responseXML.xml)

** Create required data model objects.
m.oShRsp = CREATEOBJECT('empty')

** Get response elements
m.oRps = m.oDoc.documentElement.selectNodes('/search:response')

FOR m.I = 0 TO m.oRps.length - 1
m.oRsp = m.oRps.item(m.I)
= ADDPROPERTY(m.oShRsp,'total',INT(VAL(m.oRsp.getAtt ribute('total'))))
= ADDPROPERTY(m.oShRsp,'start',INT(VAL(m.oRsp.getAtt ribute('start'))))
= ADDPROPERTY(m.oShRsp,'pgLen',INT(VAL(m.oRsp.getAtt ribute('page-length'))))

INSERT INTO xQuery (index,match) VALUES (-1,'Results = ' + TRANSFORM(m.oShRsp.total))
** Get serch results element node list list.
LOCAL oRts as MSXML2.IXMLDOMNodeList,oRst as MSXML2.IXMLDOMElement

** Create required data model objects.
m.oShRst = CREATEOBJECT('empty')
m.oRts = m.oRsp.selectNodes('./search:result')
FOR EACH m.oRst IN m.oRts
= ADDPROPERTY(m.oShRst,'index',INT(VAL(m.oRst.getAtt ribute('index'))))
m.K = AT(':/',m.oRst.getAttribute('uri'))
= ADDPROPERTY(m.oShRst,'uri',SUBSTR(m.oRst.getAttrib ute('uri'),m.K-1))
= ADDPROPERTY(m.oShRst,'docName',JUSTSTEM(m.oShRst.u ri))
= ADDPROPERTY(m.oShRst,'score',INT(VAL(m.oRst.getAtt ribute('score'))))
= ADDPROPERTY(m.oShRst,'confidence',VAL(m.oRst.getAt tribute('confidence')))
= ADDPROPERTY(m.oShRst,'fitness',INT(VAL(m.oRst.getA ttribute('fitness'))))

** From current result, get search match
LOCAL oMhs as MSXML2.IXMLDOMNodeList,oMth as MSXML2.IXMLDOMElement

m.oMhs = m.oRst.selectNodes('./search:snippet/search:match')
FOR EACH m.oMth IN m.oMhs
= ADDPROPERTY(m.oShRst,'match',m.oMth.text)
= ADDPROPERTY(m.oShRst,'navpath',m.oMth.getAttribute ('path'))

** Add current search result to cursor xquery.
INSERT INTO xquery FROM NAME m.oShRst
ENDFOR
ENDFOR
ENDFOR
ENDFUNC
ENDDEFINE


I'am looking to do the same using Lianja, without the MSXML ActiveX object, but not found any class that have this king of services.
lianjasupport
2012-10-12, 09:48
If you SET DEBUG ON in the command console and then run it, the debug.txt file in the lianja\beta directory should let you see what the method signatures are and what it causing the issue.

i agree it would be nice to have an xmlparser class exposed as standard that is cross platform and does not use activex which is windows specific.
lianjasupport
2012-10-12, 09:51
Also if you

SET DEBUGOUT ON

and put

DEBUGOUT "some message"

these will be included in the debug,txt file also which should help to track the problem down.
lianjasupport
2012-10-12, 09:54
Ah... Looking at your code further you can't use foreach on activex collections like that in Lianja you must get each item by index e.g item( n ).
pablokuster@gmail.com
2012-10-13, 02:57
i agree it would be nice to have an xmlparser class exposed as standard that is cross platform and does not use activex which is windows specific.

Yes we think the same, implement a multi platform XML PE, may be, is not the best solution, but what about some integration that can be less difficult. We ask for solutions like XML adapter that load XML without any kind of validation, because the server that receives the request and produce the response make this work in the model of web services REST or SOAP, and load this data in one or more specified tables.

This solution require only a class with the functionality of make a http request to a server and the necessary events and methods to control when the response arrive (may be with asynchronous request).



<invoice date="" number="">
<customer country="" language="">
<id>nnn</id>
<name>
<first>aaaaa</first>
<middle>bbbb</middle>
<last>cccccc</last>
</name>
<invoiceAddress></invoiceAddress>
<shipAdress></shipAdress>
<!-- other data -->>
</customer>
<detail>
<item="" desc="" quantity="" price="" ......>
<item="" desc="" quantity="" price="" ......>
<item="" desc="" quantity="" price="" ......>
<detail>
</invoice>


After this, became the XML adapter that describe how to load data in each table, es: customer and invoice, that define the schema of data to load and the binding of each column against corresponding data into XML document using xpath, relations that can be 1:1 (customer data) or 1:n (invoice details)

We continued to try with msxml to go through this paradigm, but we propose to lianja developer team to consider the possibility of integrate a solution of this kind, in today days very useful.

TIA.
barrymavin
2012-10-13, 03:02
Ok then submit an enhancement request on lianja.com. We use an XML parser internally maybe we can expose it as a standard system class.
barrymavin
2012-10-13, 03:05
I'd also like to point out that we do in fact support loading tables from XML in our SQL but the format of that XML has to be in Microsoft ADO format. There are also built-in functions that will map XML to and from an object.
pablokuster@gmail.com
2012-10-13, 03:49
Thank very much barry we write the request.

We try your useful suggestions about the debug output activation and the debug.txt ouput:

Error reported from file lib_exp1.c line 499 n=42
cmdline=m.nPageLength
Error reported from file lib_exp1.c line 499 n=42
cmdline=m.

nPageLenght and nStart are parameters passed to the ml_search method, but no have idea what is happen, can you give us some help :)

Tia.
barrymavin
2012-10-13, 04:01
have a look in debug_client.txt
pablokuster@gmail.com
2012-10-13, 05:03
Follow the relevant content of debug_client.txt

Class Activex Object OHT(145406800,249964368) Method: name='open'
File ..\..\rco_activex.cpp at line 620 method=open(QString,QString,QVariant,QVariant,QVar iant) args=5
File ..\..\dao.cpp at line 948 unknown type=0
File ..\..\rco_activex.cpp at line 626 type=U v=
Class Activex Object MSXML2.XMLHTTP.6.0(145410640,249964368) Method: name='send'
File ..\..\rco_activex.cpp at line 620 method=send() args=0
File ..\..\dao.cpp at line 948 unknown type=0
File ..\..\rco_activex.cpp at line 626 type=U v=
Class Activex Object OHT(145406800,249964368) GetProperty: name='readyState'
File ..\..\rco_activex.cpp at line 721 getProperty() name=readystate value=1 type=2 typename=int
File ..\..\rco_activex.cpp at line 726 getProperty() name=readystate value=1 type=2 typename=int value.type=N

and the debug.txt:

Error reported from file lib_exp1.c line 499 n=42
cmdline=m.nPageLength
Error reported from file lib_exp1.c line 499 n=42
cmdline=m.nStart

the tho variables (m.nPageLength,m.nStart) are parameters received in the method sh_search of my class:

FUNCTION sh_search(cQuery as String,nStart as Integer,nPageLength as Integer)

and not have nothing to do with the open or send methods of MSXML2.XMLHTTP class.

looking at debug_client.txt the only relevant content that I see are relative previous open and send methods.

tia for your help.

note: let me know if you need all the content of client_debug.txt
barrymavin
2012-10-13, 05:20
Yes, just to explain.

When DEBUG is ON it lets us (and you) see how lianja is introspecting the ActiveX.

Lianja looks into the ActiveX and determines the method and property signatures and maps them from VFP to the correct case sensitive ones, and also marshals the parameters that you pass to them.

Thats what all that mumbo-jumbo is.
HankFay
2012-10-13, 20:34
I'd also like to point out that we do in fact support loading tables from XML in our SQL but the format of that XML has to be in Microsoft ADO format. There are also built-in functions that will map XML to and from an object.

When using Copy to create them xml

use mytable
copy to "myfile.xml" xml

is the xml that is produced in the Microsoft ADO format?

tia,

Hank
lianjasupport
2012-10-13, 21:52
Yes it is if you:

SET XMLFORMAT TO ADO

Which I believe is the default.
HankFay
2012-10-14, 13:28
Yes it is if you:

SET XMLFORMAT TO ADO

Which I believe is the default.

The Docs indicate that the SET XMLFORMAT command applies only to the Select ... SAVE AS XML

Question: does SET XMLFORMAT work with any of the other commands for importing/exporting data as XML?

tia,

Hank
barrymavin
2012-10-14, 20:03
SET XMLFORMAT TO ADO is the default in Lianja.

So the COPY command produces XML in Microsoft ADO format. If you execute the following commands in the console workspace you can see the xml output.



open database southwind
use example
copy to export xml
type export.xml
barrymavin
2012-10-29, 02:24
Please look at this link (http://www.lianja.com/community/showthread.php?755-Parsing-big-data-JSON-and-XML-content-in-Lianja&p=2858#post2858) which explains how to parse big data in JSON or XML.