Difference between revisions of "Performance/load performance analysis"
Zengliangjun (talk | contribs) |
Zengliangjun (talk | contribs) |
||
(6 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
= A group of test data = | = A group of test data = | ||
− | == Interesting contrast analysis data SAX parser processing == | + | == Interesting contrast analysis data from SAX parser processing == |
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''Start OOo writer, read a document which has a 3.9 MB "content.xml" '''</center> | ||
− | + | |- | |
| ODF handler | | ODF handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 33: | Line 34: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">6529</div> | + | | <div align="right">'''6529'''</div> |
− | | <div align="right">6560</div> | + | | <div align="right">'''6560'''</div> |
− | | <div align="right">6475</div> | + | | <div align="right">'''6475'''</div> |
− | | <div align="right">5934</div> | + | | <div align="right">'''5934'''</div> |
− | | <div align="right">6738</div> | + | | <div align="right">'''6738'''</div> |
− | | <div align="right">6447.2</div> | + | | <div align="right">'''''6447.2'''''</div> |
|- | |- | ||
Line 60: | Line 61: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">6453</div> | + | | <div align="right">'''6453'''</div> |
− | | <div align="right">5568</div> | + | | <div align="right">'''5568'''</div> |
− | | <div align="right">6472</div> | + | | <div align="right">'''6472'''</div> |
− | | <div align="right">6434</div> | + | | <div align="right">'''6434'''</div> |
− | | <div align="right">6436</div> | + | | <div align="right">'''6436'''</div> |
− | | <div align="right">6272.6</div> | + | | <div align="right">'''''6272.6'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler read the sample "content.xml" '''</center> | ||
− | + | |- | |
| Empty handler | | Empty handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 100: | Line 102: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">324</div> | + | | <div align="right">'''324'''</div> |
− | | <div align="right">332</div> | + | | <div align="right">'''332'''</div> |
− | | <div align="right">326</div> | + | | <div align="right">'''326'''</div> |
− | | <div align="right">340</div> | + | | <div align="right">'''340'''</div> |
− | | <div align="right">288</div> | + | | <div align="right">'''288'''</div> |
− | | <div align="right">322</div> | + | | <div align="right">'''''322'''''</div> |
|- | |- | ||
Line 127: | Line 129: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">281</div> | + | | <div align="right">'''281'''</div> |
− | | <div align="right">282</div> | + | | <div align="right">'''282'''</div> |
− | | <div align="right">278</div> | + | | <div align="right">'''278'''</div> |
− | | <div align="right">281</div> | + | | <div align="right">'''281'''</div> |
− | | <div align="right">276</div> | + | | <div align="right">'''276'''</div> |
− | | <div align="right">279.6</div> | + | | <div align="right">'''''279.6'''''</div> |
|} | |} | ||
− | The first “start – end” | + | '''The first “start – end” contain load library cost ,the second “start – end” do not contain。''' |
All test result are warm . | All test result are warm . | ||
− | We can find the OOo processing more | + | * '''We can find the OOo processing more cost than empty XdocumentHandler。''' |
− | == Interesting contrast analysis data Dual-core processing == | + | == Interesting contrast analysis data from Dual-core processing == |
The test with 912 xml files ( total size 44.2 MB ) | The test with 912 xml files ( total size 44.2 MB ) | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and single thread process these files.'''</center> | ||
− | + | |- | |
| Single thread | | Single thread | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 176: | Line 179: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">3575</div> | + | | <div align="right">'''3575'''</div> |
− | | <div align="right">3520</div> | + | | <div align="right">'''3520'''</div> |
− | | <div align="right">3471</div> | + | | <div align="right">'''3471'''</div> |
− | | <div align="right">3515</div> | + | | <div align="right">'''3515'''</div> |
− | | <div align="right">3463</div> | + | | <div align="right">'''3463'''</div> |
− | | <div align="right">3508.8</div> | + | | <div align="right">'''''3508.8'''''</div> |
|- | |- | ||
Line 203: | Line 206: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">3472</div> | + | | <div align="right">'''3472'''</div> |
− | | <div align="right">3407</div> | + | | <div align="right">'''3407'''</div> |
− | | <div align="right">3424</div> | + | | <div align="right">'''3424'''</div> |
− | | <div align="right">3487</div> | + | | <div align="right">'''3487'''</div> |
− | | <div align="right">3405</div> | + | | <div align="right">'''3405'''</div> |
− | | <div align="right">3439</div> | + | | <div align="right">'''''3439'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and one file one thread process these files.'''</center> | ||
− | + | |- | |
| <center>'''one file one thread'''</center> | | <center>'''one file one thread'''</center> | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 243: | Line 247: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">2678</div> | + | | <div align="right">'''2678'''</div> |
− | | <div align="right">2737</div> | + | | <div align="right">'''2737'''</div> |
− | | <div align="right">2532</div> | + | | <div align="right">'''2532'''</div> |
− | | <div align="right">2797</div> | + | | <div align="right">'''2797'''</div> |
− | | <div align="right">2100</div> | + | | <div align="right">'''2100'''</div> |
− | | <div align="right">2568.8</div> | + | | <div align="right">'''''2568.8'''''</div> |
|- | |- | ||
Line 270: | Line 274: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">2622</div> | + | | <div align="right">'''2622'''</div> |
− | | <div align="right">2707</div> | + | | <div align="right">'''2707'''</div> |
− | | <div align="right">2664</div> | + | | <div align="right">'''2664'''</div> |
− | | <div align="right">2347</div> | + | | <div align="right">'''2347'''</div> |
− | | <div align="right">2683</div> | + | | <div align="right">'''2683'''</div> |
− | | <div align="right">2604.6</div> | + | | <div align="right">'''''2604.6'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and two thread process these files.'''</center> | ||
− | + | |- | |
| Two thread | | Two thread | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 310: | Line 315: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">2287</div> | + | | <div align="right">'''2287'''</div> |
− | | <div align="right">2303</div> | + | | <div align="right">'''2303'''</div> |
− | | <div align="right">2327</div> | + | | <div align="right">'''2327'''</div> |
− | | <div align="right">2347</div> | + | | <div align="right">'''2347'''</div> |
− | | <div align="right">2381</div> | + | | <div align="right">'''2381'''</div> |
− | | <div align="right">2329</div> | + | | <div align="right">'''''2329'''''</div> |
|- | |- | ||
Line 337: | Line 342: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">2254</div> | + | | <div align="right">'''2254'''</div> |
− | | <div align="right">2279</div> | + | | <div align="right">'''2279'''</div> |
− | | <div align="right">2266</div> | + | | <div align="right">'''2266'''</div> |
− | | <div align="right">2262</div> | + | | <div align="right">'''2262'''</div> |
− | | <div align="right">2320</div> | + | | <div align="right">'''2320'''</div> |
− | | <div align="right">2276.2</div> | + | | <div align="right">'''''2276.2'''''</div> |
|} | |} | ||
− | == Interesting contrast analysis data Mononuclear processing == | + | == Interesting contrast analysis data from Mononuclear processing == |
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and single thread process these files.'''</center> | ||
− | + | |- | |
| Single thread | | Single thread | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 378: | Line 384: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">5469</div> | + | | <div align="right">'''5469'''</div> |
− | | <div align="right">5469</div> | + | | <div align="right">'''5469'''</div> |
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">5063</div> | + | | <div align="right">'''5063'''</div> |
− | | <div align="right">5200.2</div> | + | | <div align="right">'''''5200.2'''''</div> |
|- | |- | ||
Line 405: | Line 411: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">4954</div> | + | | <div align="right">'''4954'''</div> |
− | | <div align="right">5172</div> | + | | <div align="right">'''5172'''</div> |
− | | <div align="right">4953</div> | + | | <div align="right">'''4953'''</div> |
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">4968</div> | + | | <div align="right">'''4968'''</div> |
− | | <div align="right">5009.4</div> | + | | <div align="right">'''''5009.4'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and one file one thread process these files.'''</center> | ||
− | + | |- | |
| <center>'''one file one thread'''</center> | | <center>'''one file one thread'''</center> | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 445: | Line 452: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5328</div> | + | | <div align="right">'''5328'''</div> |
− | | <div align="right">5344</div> | + | | <div align="right">'''5344'''</div> |
− | | <div align="right">5375</div> | + | | <div align="right">'''5375'''</div> |
− | | <div align="right">5734</div> | + | | <div align="right">'''5734'''</div> |
− | | <div align="right">5453</div> | + | | <div align="right">'''5453'''</div> |
− | | <div align="right">5446.8</div> | + | | <div align="right">'''''5446.8'''''</div> |
|- | |- | ||
Line 472: | Line 479: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5328</div> | + | | <div align="right">'''5328'''</div> |
− | | <div align="right">5381</div> | + | | <div align="right">'''5381'''</div> |
− | | <div align="right">5281</div> | + | | <div align="right">'''5281'''</div> |
− | | <div align="right">5656</div> | + | | <div align="right">'''5656'''</div> |
− | | <div align="right">5281</div> | + | | <div align="right">'''5281'''</div> |
− | | <div align="right">5385.4</div> | + | | <div align="right">'''''5385.4'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and two thread process these files.'''</center> | ||
− | + | |- | |
| Two thread | | Two thread | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 512: | Line 520: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5063</div> | + | | <div align="right">'''5063'''</div> |
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">5469</div> | + | | <div align="right">'''5469'''</div> |
− | | <div align="right">5469</div> | + | | <div align="right">'''5469'''</div> |
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">5200</div> | + | | <div align="right">'''''5200'''''</div> |
|- | |- | ||
Line 539: | Line 547: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">4968</div> | + | | <div align="right">'''4968'''</div> |
− | | <div align="right">5000</div> | + | | <div align="right">'''5000'''</div> |
− | | <div align="right">4953</div> | + | | <div align="right">'''4953'''</div> |
− | | <div align="right">5172</div> | + | | <div align="right">'''5172'''</div> |
− | | <div align="right">4954</div> | + | | <div align="right">'''4954'''</div> |
− | | <div align="right">5009.4</div> | + | | <div align="right">'''''5009.4'''''</div> |
|} | |} | ||
= Overview = | = Overview = | ||
− | Through the | + | == Through the first group data we can get a conclusion: == |
− | + | * ''' As load odf, OOo process that is great proportion .''' | |
− | |||
− | |||
− | |||
− | |||
− | Through the | + | == Through the late two sets data we can get the conclusions: == |
− | + | * ''' As single-core, one task divide into multiple small tasks;these are same cost, processing with single thread, one task one thread and two thread.( one task one thread performance relatively low) ''' | |
− | + | * ''' As dual-core, one task divide into multiple small tasks;that is close to 0.7-0.8 compared one task one thread, that is close to 0.66 compared two threads process; to single thread. ''' | |
= How to do = | = How to do = | ||
− | How can we improve ODF loading performance. | + | == How can we improve ODF loading performance.== |
− | + | * ''' As multi-core cpu, Can we split “content.xml” processing to some small tasks; process with multi-thread and merger the result. ''' | |
− | + | * ''' As Single-core, Can we split “content.xml” processing to some small tasks; asynchronous processing.''' | |
− | <nowiki>* | + | <nowiki>* Now, we don't discuss the xml parser. </nowiki> |
+ | == How can we split “content.xml” processing to some small tasks: == | ||
− | + | * ''' Split “content.xml” content. ''' | |
− | + | * ''' Colloect the result with sax parser, and Split the result ''' | |
− | |||
− | |||
= Description = | = Description = | ||
Line 579: | Line 582: | ||
Change the document structure, complex nodules defined as a reference link to a single file,the main file contains only simple structure. | Change the document structure, complex nodules defined as a reference link to a single file,the main file contains only simple structure. | ||
− | It will produce the ODF layout : | + | It will produce the ODF layout: |
− | content.xml | + | <nowiki>content.xml |
− | + | complex/fragment1.xml | |
− | + | /fragment2.xml | |
− | + | /fragment3.xml | |
− | + | /fragment4.xml | |
− | + | …</nowiki> | |
− | And “content.xml” will be : | + | And “content.xml” will be: |
− | … | + | <nowiki>… |
− | + | <table:table xlink:href= “complex/fragment1.xml” /> | |
− | + | …</nowiki> | |
− | |||
− | |||
− | |||
+ | * I think it will produce the same performance optimization when save documentation, this way. | ||
== Colloect the result with sax parser, and Split that == | == Colloect the result with sax parser, and Split that == | ||
Line 602: | Line 603: | ||
− | <center> | + | {| class="prettytable" |
+ | | colspan="7" | <center>A simple application it implement a XdocumentHandler that colloect the SAX parser result , use UCB read thest files</center> | ||
− | + | |- | |
− | + | | <center>Colloect the SAX parser result</center> | |
− | | <center> | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
− | | | ||
| | | | ||
Line 634: | Line 635: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5677</div> | + | | <div align="right">'''5677'''</div> |
− | | <div align="right">5491</div> | + | | <div align="right">'''5491'''</div> |
− | | <div align="right">5181</div> | + | | <div align="right">'''5181'''</div> |
− | | <div align="right">5812</div> | + | | <div align="right">'''5812'''</div> |
− | | <div align="right">5516</div> | + | | <div align="right">'''5516'''</div> |
− | | <div align="right">5695.4</div> | + | | <div align="right">'''''5695.4'''''</div> |
|- | |- | ||
Line 661: | Line 662: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5108</div> | + | | <div align="right">'''5108'''</div> |
− | | <div align="right">5175</div> | + | | <div align="right">'''5175'''</div> |
− | | <div align="right">5097</div> | + | | <div align="right">'''5097'''</div> |
− | | <div align="right">5126</div> | + | | <div align="right">'''5126'''</div> |
− | | <div align="right">5267</div> | + | | <div align="right">'''5267'''</div> |
− | | <div align="right">5154.6</div> | + | | <div align="right">'''''5154.6'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a empty XdocumentHandler , use UCB and single thread process these files.'''</center> | ||
− | + | |- | |
| Empty handler | | Empty handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 701: | Line 703: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">3773</div> | + | | <div align="right">'''3773'''</div> |
− | | <div align="right">3752</div> | + | | <div align="right">'''3752'''</div> |
− | | <div align="right">3895</div> | + | | <div align="right">'''3895'''</div> |
− | | <div align="right">4030</div> | + | | <div align="right">'''4030'''</div> |
− | | <div align="right">3686</div> | + | | <div align="right">'''3686'''</div> |
− | | <div align="right">3821.2</div> | + | | <div align="right">'''''3821.2'''''</div> |
|- | |- | ||
Line 728: | Line 730: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">3542</div> | + | | <div align="right">'''3542'''</div> |
− | | <div align="right">3680</div> | + | | <div align="right">'''3680'''</div> |
− | | <div align="right">3924</div> | + | | <div align="right">'''3924'''</div> |
− | | <div align="right">3732</div> | + | | <div align="right">'''3732'''</div> |
− | | <div align="right">3762</div> | + | | <div align="right">'''3762'''</div> |
− | | <div align="right">3728</div> | + | | <div align="right">'''''3728'''''</div> |
|} | |} | ||
We can see it is less cost that colloect the SAX parser result. So we can split that throught some rules and multi-thread to process . | We can see it is less cost that colloect the SAX parser result. So we can split that throught some rules and multi-thread to process . | ||
+ | |||
+ | == Implement Description == | ||
+ | It has two policies which split processing “content.xml”task to some small tasks: | ||
+ | |||
+ | a) Split “content.xml”to many sub files. | ||
+ | |||
+ | b) Colloect the result(attribute & content ) from sax parser, and split the result to many parts in some rules. | ||
+ | |||
+ | struct State | ||
+ | { | ||
+ | rtl_uString* mp_elementName; // current element tag name can replace to one flag | ||
+ | xml::sax::XAttributeList* mp_attributes; // current element's attributes | ||
+ | rtl_uString* mp_elementContent; // current element's content | ||
+ | |||
+ | sal_uInt32 m_local; // It's position at vector | ||
+ | sal_uInt32 m_distance; // It's distance from parent element | ||
+ | sal_uInt32 m_count; // It's count of all sub elements | ||
+ | }; | ||
+ | |||
+ | <nowiki>typedef std::vector< State* > StateVectors;</nowiki> | ||
+ | |||
+ | From compared above multi-thread data, situation single-core / multi-core increase performance is different. | ||
+ | |||
+ | So I think it can consider asynchronous processing as single-core, it can consider parallel processing as multi-core. | ||
+ | |||
+ | It is 2 policies X 2 situations processing description( Of couse it is my imagines ): | ||
+ | |||
+ | === Single Core + Many Fragments Asynchronism Process === | ||
+ | [[Image:Single_Core_%2B_Many_Fragments_Asynchronism_Process.PNG]] | ||
+ | === Single Core + split SAX result Asynchronism Process === | ||
+ | [[Image:Single_Core_%2B_split_SAX_result_Asynchronism_Process.PNG]] | ||
+ | |||
+ | === Multi Core + Many fragments Parallel Process === | ||
+ | [[Image:Multi_Core_%2B_Many_fragments_Parallel_Process.PNG]] | ||
+ | === Multi-Core + split SAX result Parallel Process === | ||
+ | [[Image:Multi_Core_%2B_split_SAX_result_Parallel_Process.PNG]] | ||
== Key points == | == Key points == | ||
Line 743: | Line 781: | ||
= Appendix = | = Appendix = | ||
== Here is simple test, with no other considerations, just to test the feasibility of the technology. == | == Here is simple test, with no other considerations, just to test the feasibility of the technology. == | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a XdocumentHandler that colloect the SAX parser result , use UCB read thest files'''</center> | ||
− | + | |- | |
− | | <center> | + | | <center>Colloect the SAX parser result</center> |
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 775: | Line 814: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5527</div> | + | | <div align="right">'''5527'''</div> |
− | | <div align="right">5534</div> | + | | <div align="right">'''5534'''</div> |
− | | <div align="right">5521</div> | + | | <div align="right">'''5521'''</div> |
− | | <div align="right">5761</div> | + | | <div align="right">'''5761'''</div> |
− | | <div align="right">5535</div> | + | | <div align="right">'''5535'''</div> |
− | | <div align="right">5575.6</div> | + | | <div align="right">'''''5575.6'''''</div> |
|- | |- | ||
Line 802: | Line 841: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5392</div> | + | | <div align="right">'''5392'''</div> |
− | | <div align="right">5530</div> | + | | <div align="right">'''5530'''</div> |
− | | <div align="right">5518</div> | + | | <div align="right">'''5518'''</div> |
− | | <div align="right">5477</div> | + | | <div align="right">'''5477'''</div> |
− | | <div align="right">5414</div> | + | | <div align="right">'''5414'''</div> |
− | | <div align="right">5466.2</div> | + | | <div align="right">'''''5466.2'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''A simple application it implement a XdocumentHandler that colloect the SAX parser result , use UCB read thest files, and one thread process one SAX parser result.'''</center> | ||
− | + | |- | |
− | |||
− | |||
− | |||
− | |||
− | | | ||
| | | | ||
+ | | <div align="right">No . 1</div> | ||
+ | | <div align="right">No . 2</div> | ||
+ | | <div align="right">No . 3</div> | ||
+ | | <div align="right">No . 4</div> | ||
+ | | <div align="right">No . 5</div> | ||
| | | | ||
Line 842: | Line 882: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">28291</div> | + | | <div align="right">'''28291'''</div> |
− | | <div align="right">28403</div> | + | | <div align="right">'''28403'''</div> |
− | | <div align="right">28085</div> | + | | <div align="right">'''28085'''</div> |
− | | <div align="right">27805</div> | + | | <div align="right">'''27805'''</div> |
− | | <div align="right">28034</div> | + | | <div align="right">'''28034'''</div> |
− | | <div align="right">28123.6</div> | + | | <div align="right">'''''28123.6'''''</div> |
|- | |- | ||
Line 869: | Line 909: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">27437</div> | + | | <div align="right">'''27437'''</div> |
− | | <div align="right">27610</div> | + | | <div align="right">'''27610'''</div> |
− | | <div align="right">28179</div> | + | | <div align="right">'''28179'''</div> |
− | | <div align="right">28639</div> | + | | <div align="right">'''28639'''</div> |
− | | <div align="right">28484</div> | + | | <div align="right">'''28484'''</div> |
− | | <div align="right">28069.8</div> | + | | <div align="right">'''''28069.8'''''</div> |
|} | |} | ||
== A group of large data test == | == A group of large data test == | ||
=== OOo calc “content.xml” 35.1 MB / 3.8M === | === OOo calc “content.xml” 35.1 MB / 3.8M === | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''OOo reader “content.xml” '''</center> | ||
− | + | |- | |
| ODF handler | | ODF handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 911: | Line 952: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">31661</div> | + | | <div align="right">'''31661'''</div> |
− | | <div align="right">31048</div> | + | | <div align="right">'''31048'''</div> |
− | | <div align="right">30126</div> | + | | <div align="right">'''30126'''</div> |
− | | <div align="right">27885</div> | + | | <div align="right">'''27885'''</div> |
− | | <div align="right">28865</div> | + | | <div align="right">'''28865'''</div> |
− | | <div align="right">29917</div> | + | | <div align="right">'''''29917'''''</div> |
|- | |- | ||
Line 938: | Line 979: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">32088</div> | + | | <div align="right">'''32088'''</div> |
− | | <div align="right">32752</div> | + | | <div align="right">'''32752'''</div> |
− | | <div align="right">30918</div> | + | | <div align="right">'''30918'''</div> |
− | | <div align="right">29477</div> | + | | <div align="right">'''29477'''</div> |
− | | <div align="right">28593</div> | + | | <div align="right">'''28593'''</div> |
− | | <div align="right">30765.6</div> | + | | <div align="right">'''''30765.6'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''“XdocumentHandler” empty implement reader “content.xml” '''</center> | ||
− | + | |- | |
| Empty handler | | Empty handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 978: | Line 1,020: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">2066</div> | + | | <div align="right">'''2066'''</div> |
− | | <div align="right">2162</div> | + | | <div align="right">'''2162'''</div> |
− | | <div align="right">2105</div> | + | | <div align="right">'''2105'''</div> |
− | | <div align="right">2091</div> | + | | <div align="right">'''2091'''</div> |
− | | <div align="right">2068</div> | + | | <div align="right">'''2068'''</div> |
− | | <div align="right">2098.4</div> | + | | <div align="right">'''''2098.4'''''</div> |
|- | |- | ||
Line 1,005: | Line 1,047: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">1988</div> | + | | <div align="right">'''1988'''</div> |
− | | <div align="right">2574</div> | + | | <div align="right">'''2574'''</div> |
− | | <div align="right">1988</div> | + | | <div align="right">'''1988'''</div> |
− | | <div align="right">1982</div> | + | | <div align="right">'''1982'''</div> |
− | | <div align="right">2212</div> | + | | <div align="right">'''2212'''</div> |
− | | <div align="right">2148.8</div> | + | | <div align="right">'''''2148.8'''''</div> |
|} | |} | ||
=== OOo writer “content.xml” 62.6 MB / 2.0M === | === OOo writer “content.xml” 62.6 MB / 2.0M === | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''OOo reader “content.xml” '''</center> | ||
− | + | |- | |
| ODF handler | | ODF handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 1,046: | Line 1,089: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">102675</div> | + | | <div align="right">'''102675'''</div> |
− | | <div align="right">102669</div> | + | | <div align="right">'''102669'''</div> |
− | | <div align="right">102757</div> | + | | <div align="right">'''102757'''</div> |
− | | <div align="right">101872</div> | + | | <div align="right">'''101872'''</div> |
− | | <div align="right">101725</div> | + | | <div align="right">'''101725'''</div> |
− | | <div align="right">102339.6</div> | + | | <div align="right">'''''102339.6'''''</div> |
|- | |- | ||
Line 1,073: | Line 1,116: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">103322</div> | + | | <div align="right">'''103322'''</div> |
− | | <div align="right">102849</div> | + | | <div align="right">'''102849'''</div> |
− | | <div align="right">102837</div> | + | | <div align="right">'''102837'''</div> |
− | | <div align="right">102281</div> | + | | <div align="right">'''102281'''</div> |
− | | <div align="right">102314</div> | + | | <div align="right">'''102314'''</div> |
− | | <div align="right">102720.6</div> | + | | <div align="right">'''''102720.6'''''</div> |
|} | |} | ||
− | |||
+ | {| class="prettytable" | ||
+ | | colspan="7" | <center>'''“XdocumentHandler” empty implement reader “content.xml” '''</center> | ||
− | + | |- | |
| Empty handler | | Empty handler | ||
− | | | + | | <div align="right">No . 1</div> |
− | | | + | | <div align="right">No . 2</div> |
− | | | + | | <div align="right">No . 3</div> |
− | | | + | | <div align="right">No . 4</div> |
− | | | + | | <div align="right">No . 5</div> |
| | | | ||
Line 1,113: | Line 1,157: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">4805</div> | + | | <div align="right">'''4805'''</div> |
− | | <div align="right">5536</div> | + | | <div align="right">'''5536'''</div> |
− | | <div align="right">5534</div> | + | | <div align="right">'''5534'''</div> |
− | | <div align="right">5635</div> | + | | <div align="right">'''5635'''</div> |
− | | <div align="right">5443</div> | + | | <div align="right">'''5443'''</div> |
− | | <div align="right">5390.6</div> | + | | <div align="right">'''''5390.6'''''</div> |
|- | |- | ||
Line 1,140: | Line 1,184: | ||
|- | |- | ||
| | | | ||
− | | <div align="right">5074</div> | + | | <div align="right">'''5074'''</div> |
− | | <div align="right">6220</div> | + | | <div align="right">'''6220'''</div> |
− | | <div align="right">5734</div> | + | | <div align="right">'''5734'''</div> |
− | | <div align="right">5642</div> | + | | <div align="right">'''5642'''</div> |
− | | <div align="right">5322</div> | + | | <div align="right">'''5322'''</div> |
− | | <div align="right">5598.4</div> | + | | <div align="right">'''''5598.4'''''</div> |
|} | |} | ||
+ | |||
+ | |||
[[Category:Performance]] | [[Category:Performance]] |
Latest revision as of 05:14, 8 December 2009
A group of test data
Interesting contrast analysis data from SAX parser processing
ODF handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 12759
|
11805
|
13233
|
11312
|
13279
|
|
Start | 6230
|
5245
|
6758
|
5378
|
6541
|
|
6529
|
6560
|
6475
|
5934
|
6738
|
6447.2
| |
End | 31305
|
36262
|
30083
|
28452
|
28584
|
|
Start | 24852
|
30694
|
23611
|
22018
|
22148
|
|
6453
|
5568
|
6472
|
6434
|
6436
|
6272.6
|
Empty handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 354
|
362
|
355
|
370
|
312
|
|
Start | 30
|
30
|
29
|
30
|
24
|
|
324
|
332
|
326
|
340
|
288
|
322
| |
End | 635
|
644
|
633
|
651
|
588
|
|
Start | 354
|
362
|
355
|
370
|
312
|
|
281
|
282
|
278
|
281
|
276
|
279.6
|
The first “start – end” contain load library cost ,the second “start – end” do not contain。
All test result are warm .
- We can find the OOo processing more cost than empty XdocumentHandler。
Interesting contrast analysis data from Dual-core processing
The test with 912 xml files ( total size 44.2 MB )
Single thread | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 003604
|
003549
|
003500
|
003538
|
003492
|
|
Start | 000029
|
000029
|
000029
|
000023
|
000029
|
|
3575
|
3520
|
3471
|
3515
|
3463
|
3508.8
| |
End | 007076
|
006957
|
006924
|
007025
|
006897
|
|
Start | 003604
|
003550
|
003500
|
003538
|
003492
|
|
3472
|
3407
|
3424
|
3487
|
3405
|
3439
|
No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
||
End | 002696
|
002763
|
002560
|
002827
|
002133
|
|
Start | 000018
|
000026
|
000028
|
000030
|
000033
|
|
2678
|
2737
|
2532
|
2797
|
2100
|
2568.8
| |
End | 005318
|
005471
|
005224
|
005174
|
004816
|
|
Start | 002696
|
002764
|
002560
|
002827
|
002133
|
|
2622
|
2707
|
2664
|
2347
|
2683
|
2604.6
|
Two thread | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 002314
|
002332
|
002356
|
002377
|
002411
|
|
Start | 000027
|
000029
|
000029
|
000030
|
000030
|
|
2287
|
2303
|
2327
|
2347
|
2381
|
2329
| |
End | 004568
|
004611
|
004622
|
004639
|
004731
|
|
Start | 002314
|
002332
|
002356
|
002377
|
002411
|
|
2254
|
2279
|
2266
|
2262
|
2320
|
2276.2
|
Interesting contrast analysis data from Mononuclear processing
Single thread | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 005062
|
005531
|
005516
|
005062
|
005110
|
|
Start | 000062
|
000062
|
000047
|
000062
|
000047
|
|
5000
|
5469
|
5469
|
5000
|
5063
|
5200.2
| |
End | 010016
|
010703
|
010469
|
010062
|
010078
|
|
Start | 005062
|
005531
|
005516
|
005062
|
005110
|
|
4954
|
5172
|
4953
|
5000
|
4968
|
5009.4
|
No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
||
End | 005391
|
005391
|
005422
|
005797
|
005516
|
|
Start | 000063
|
000047
|
000047
|
000063
|
000063
|
|
5328
|
5344
|
5375
|
5734
|
5453
|
5446.8
| |
End | 010719
|
010672
|
010703
|
011453
|
010797
|
|
Start | 005391
|
005391
|
005422
|
005797
|
005516
|
|
5328
|
5381
|
5281
|
5656
|
5281
|
5385.4
|
Two thread | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 005110
|
005062
|
005516
|
005531
|
005062
|
|
Start | 000047
|
000062
|
000047
|
000062
|
000062
|
|
5063
|
5000
|
5469
|
5469
|
5000
|
5200
| |
End | 010078
|
010062
|
010469
|
010703
|
010016
|
|
Start | 005110
|
005062
|
005516
|
005531
|
005062
|
|
4968
|
5000
|
4953
|
5172
|
4954
|
5009.4
|
Overview
Through the first group data we can get a conclusion:
- As load odf, OOo process that is great proportion .
Through the late two sets data we can get the conclusions:
- As single-core, one task divide into multiple small tasks;these are same cost, processing with single thread, one task one thread and two thread.( one task one thread performance relatively low)
- As dual-core, one task divide into multiple small tasks;that is close to 0.7-0.8 compared one task one thread, that is close to 0.66 compared two threads process; to single thread.
How to do
How can we improve ODF loading performance.
- As multi-core cpu, Can we split “content.xml” processing to some small tasks; process with multi-thread and merger the result.
- As Single-core, Can we split “content.xml” processing to some small tasks; asynchronous processing.
* Now, we don't discuss the xml parser.
How can we split “content.xml” processing to some small tasks:
- Split “content.xml” content.
- Colloect the result with sax parser, and Split the result
Description
Split “content.xml” content
Change the document structure, complex nodules defined as a reference link to a single file,the main file contains only simple structure.
It will produce the ODF layout:
content.xml complex/fragment1.xml /fragment2.xml /fragment3.xml /fragment4.xml …
And “content.xml” will be:
… <table:table xlink:href= “complex/fragment1.xml” /> …
- I think it will produce the same performance optimization when save documentation, this way.
Colloect the result with sax parser, and Split that
From the above test data, SAX process with relatively less cost itself. We can colloect SAX parser result, and split it to multi part.
No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
||
End | 005706
|
005510
|
005200
|
005824
|
005527
|
|
Start | 000029
|
000019
|
000019
|
000012
|
000011
|
|
5677
|
5491
|
5181
|
5812
|
5516
|
5695.4
| |
End | 010814
|
010685
|
010297
|
010950
|
010794
|
|
Start | 005706
|
005510
|
005200
|
005824
|
005527
|
|
5108
|
5175
|
5097
|
5126
|
5267
|
5154.6
|
Empty handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 003788
|
003763
|
003908
|
004044
|
003702
|
|
Start | 000015
|
000011
|
000013
|
000014
|
000016
|
|
3773
|
3752
|
3895
|
4030
|
3686
|
3821.2
| |
End | 008330
|
007444
|
007836
|
007776
|
007465
|
|
Start | 003788
|
003764
|
003908
|
004044
|
003703
|
|
3542
|
3680
|
3924
|
3732
|
3762
|
3728
|
We can see it is less cost that colloect the SAX parser result. So we can split that throught some rules and multi-thread to process .
Implement Description
It has two policies which split processing “content.xml”task to some small tasks:
a) Split “content.xml”to many sub files.
b) Colloect the result(attribute & content ) from sax parser, and split the result to many parts in some rules.
struct State { rtl_uString* mp_elementName; // current element tag name can replace to one flag xml::sax::XAttributeList* mp_attributes; // current element's attributes rtl_uString* mp_elementContent; // current element's content
sal_uInt32 m_local; // It's position at vector sal_uInt32 m_distance; // It's distance from parent element sal_uInt32 m_count; // It's count of all sub elements };
typedef std::vector< State* > StateVectors;
From compared above multi-thread data, situation single-core / multi-core increase performance is different.
So I think it can consider asynchronous processing as single-core, it can consider parallel processing as multi-core.
It is 2 policies X 2 situations processing description( Of couse it is my imagines ):
Single Core + Many Fragments Asynchronism Process
Single Core + split SAX result Asynchronism Process
Multi Core + Many fragments Parallel Process
Multi-Core + split SAX result Parallel Process
Key points
The collected information segmentation algorithm
Now for OOo process is not very clear, do n't know whether be expected. Waits further research
Appendix
Here is simple test, with no other considerations, just to test the feasibility of the technology.
No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
||
End | 005560
|
005563
|
005551
|
005791
|
005558
|
|
Start | 000033
|
000029
|
000030
|
000030
|
000023
|
|
5527
|
5534
|
5521
|
5761
|
5535
|
5575.6
| |
End | 010952
|
011093
|
011069
|
011268
|
010972
|
|
Start | 005560
|
005563
|
005551
|
005791
|
005558
|
|
5392
|
5530
|
5518
|
5477
|
5414
|
5466.2
|
No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
||
End | 028326
|
028438
|
028115
|
27835
|
28054
|
|
Start | 000035
|
000035
|
000030
|
30
|
20
|
|
28291
|
28403
|
28085
|
27805
|
28034
|
28123.6
| |
End | 055763
|
056048
|
056294
|
056474
|
056538
|
|
Start | 028326
|
028438
|
28115
|
027835
|
028054
|
|
27437
|
27610
|
28179
|
28639
|
28484
|
28069.8
|
A group of large data test
OOo calc “content.xml” 35.1 MB / 3.8M
ODF handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 38220
|
37340
|
37140
|
66316
|
35378
|
|
Start | 6559
|
6292
|
7014
|
38431
|
6513
|
|
31661
|
31048
|
30126
|
27885
|
28865
|
29917
| |
End | 79549
|
82462
|
81397
|
107576
|
873537
|
|
Start | 47461
|
49710
|
50479
|
78099
|
844944
|
|
32088
|
32752
|
30918
|
29477
|
28593
|
30765.6
|
Empty handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 2082
|
2178
|
2118
|
2106
|
2082
|
|
Start | 16
|
16
|
13
|
15
|
14
|
|
2066
|
2162
|
2105
|
2091
|
2068
|
2098.4
| |
End | 4070
|
4752
|
4106
|
4088
|
4294
|
|
Start | 2082
|
2178
|
2118
|
2106
|
2082
|
|
1988
|
2574
|
1988
|
1982
|
2212
|
2148.8
|
OOo writer “content.xml” 62.6 MB / 2.0M
ODF handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 108876
|
108729
|
108997
|
108901
|
110140
|
|
Start | 6201
|
6060
|
6240
|
7029
|
8415
|
|
102675
|
102669
|
102757
|
101872
|
101725
|
102339.6
| |
End | 229982
|
228416
|
235011
|
234573
|
233124
|
|
Start | 126660
|
125567
|
132174
|
132292
|
130810
|
|
103322
|
102849
|
102837
|
102281
|
102314
|
102720.6
|
Empty handler | No . 1
|
No . 2
|
No . 3
|
No . 4
|
No . 5
|
|
End | 4825
|
5595
|
5546
|
5658
|
6072
|
|
Start | 20
|
59
|
12
|
23
|
629
|
|
4805
|
5536
|
5534
|
5635
|
5443
|
5390.6
| |
End | 9899
|
11815
|
11280
|
11300
|
11394
|
|
Start | 4825
|
5595
|
5546
|
5658
|
6072
|
|
5074
|
6220
|
5734
|
5642
|
5322
|
5598.4
|