coldfusion - How to deserialize a very large JSON string -


i'm using below api load data our application

rec registry

as per specification, uses date in yyyy-mm-dd format input. i'm trying deserialize returned json string due large amount of data; i'm getting connection failure error.

is there better way (probably asynchronous) same? tried searching of solutions involve .net.

code far:

<cfhttp url="https://rec-registry.gov.au/rec-registry/app/api/public-register/certificate-actions"         result ="recregistry_json"          method="get" getasbinary="never">     <cfhttpparam name="date" value="#dateformat(dateadd("d",-3, now()),"yyyy-mm-dd")#" type="url"> </cfhttp>  <cfset recjson = deserializejson(recregistry_json.filecontent)>  <cfdump var="#recjson#">

edit 1 : based on adam cameron's comment, issue cfdump not able dump huge chunks of data. deserialisation works fine 20 mb worth of data.

in running code locally issue having http request timing out. solve issue added timeout parameter cfhttp tag cfsetting tag top of page. 

<cfsetting requesttimeout="9000"> <cfhttp          url="https://rec-registry.gov.au/rec-registry/app/api/public-register/certificate-actions"         result ="recregistry_json"          method="get"          getasbinary="never"          timeout="9000"          file="response.json"         path="#expandpath('/')#" >     <cfhttpparam name="date" value="#dateformat(dateadd("d",-3, now()),"yyyy-mm-dd")#" type="url"> </cfhttp>

i split functions of downloading file , parsing file did not have wait on download test parsing. note file , path settings on cfhttp tag.

not knowing how intend process data may better off getting smaller chunks such getting 1 day @ time , compiling results after being put local solution.


Comments

Post a Comment

Popular posts from this blog

Python Kivy ListView: How to delete selected ListItemButton? -

Image
i'm trying learn kivy building simple todo-list app suggested dusty phillips, author of book "creating apps in kivy". this code far: from kivy.app import app kivy.uix.boxlayout import boxlayout kivy.properties import objectproperty kivy.uix.listview import listitembutton class taskbutton(listitembutton): pass class todoroot(boxlayout): task_input = objectproperty() task_list = objectproperty() def add_task(self): self.task_list.adapter.data.extend([self.task_input.text]) self.task_list._trigger_reset_populate() def del_task(self): pass class todoapp(app): def build(self): return todoroot() if __name__ == '__main__': todoapp().run() and kv file: #: import main todo #: import listadapter kivy.adapters.listadapter.listadapter #: import listitembutton kivy.uix.listview.listitembutton todoroot: <todoroot>: orientation: "vertical" task_input: task_input_view t...
Read more

asp.net mvc 4 - A specified Include path is not valid. The EntityType '' does not declare a navigation property with the name '' -

i wonder if help. above error when call .include() . breaks when include tblmemberships . this.dbcontext.configuration.lazyloadingenabled = false; list<tblcustomerinfo> customers = this.dbcontext.tblcustomerinfoes.include("tblusers").include("tblmemberships").tolist(); the reason because navigation property between tblcustomerinfo , tblmemberships not exist. tblusers link between other 2 tables. customer -- 1:* -- user -- 1:* -- membership (sorry, can't include image reputataion < 10). my questions are: what need in order have tblmemberships included? is recommended way of retrieve data or should break 2 queries? or design totally rubbish? i using ef5, asp .net mvc 4 kindly advise. thank you. when write code this: db.parenttable .include("childtable") .include("childofchildtable"); you saying include entries childtable keyed parenttable , include entries childofchildtable keyed paren...
Read more