To the problem that finding rules in enormous data is very time - consumable and the expansibility of existed algorithms is not very good , the thesis proposes a new method to discompose large data table based on the concepts of positive region and the importance of attribute in rough set theory . existed algorithms of rule deduction can be applied directly on the tree structure obtained by partition and the times for computation will be reduced observably . validation of information entropy on the partition structure shows that the partition of data table will not lead to the loss of information , while the computing speed increases at the same time , which reflects the practicability and rationality about the partition of large data table 针对海量数据处理起来极为耗时,现有算法拓展性较差的问题,基于rough集理论中的集合正域概念以及由此定义的属性重要性概念,提出一种大型数据表分解算法,现有的规则归纳算法可直接在分解得到的树型结构上应用,将大大降低知识发现的时间,并从信息理论的角度利用信息熵概念对该分解结构进行了验证,分析了这种分解的实用性及合理性,揭示了这种分解结构在提高计算速度的同时不会损失信息量。
In view of above problems , this article studies the key technology of mass data processing system , and design a system framework point to the triangle mesh data , which is made up of millions series of triangles . this framework resolves the problem that a computer could not read all the mass data into its storage and process them through data block partion , and also reduces the space that mass data occupies in hd by data compression 针对以上问题,本文研究了海量数据处理系统的关键技术,针对由百万级数量的三角形构成的三角网格数据,设计了一个系统框架,通过数据分块解决计算机不能把海量数据全部读入内存进行处理的问题,通过数据压缩解决海量数据占用大量硬盘空间的问题。
The main works are as following : 1 . in view of the large quantity of data obtained from optical measurement equipments , we put forward the idea of mass data process system framework , which is data realm division first and then block data compression according it , we establish conceptual framework base on that , and design system structure , data stream 主要工作如下: 1 、针对光学测量设备得到的测量数据具有数据量巨大的特点,本文提出了先采用数据区域划分,再对各划分后的数据按区域压缩的海量数据处理系统框架设计思路,建立了总体框架,设计了系统结构、数据流。
海量: magnanimity数据处理: data handling; data processi ...测量数据处理: measurement and data processing大量数据处理: mass data processing批量数据处理: batch data proce ing; batch data processing海量数据多道处理: mass data multiprocessing海量数据: a huge quantity of data; huge quantities of data; mass data大量数据 海量数据: massdata海量数据存储平台: mars storage platform数据处理数据处理: data manipulation光数据处理: optical data processing批数据处理: batch data processing声数据处理: adacoustic data development数据处理: data handling; data processing◇数据处理程序 data processor; 数据处理机 data processor; datatron; data-processing machine; 数据处理技术 data processing technique; 数据处理能力 [自动化] data-handling capacity; 数据处理器 data processor; data process equipment数据处理部: data processing department; dpd data processing division数据处理机: array processor; data datatron; data handling unit; data machine; data proce ing machine; data proce machine; data proce or; data process machine; data processing equipment; data processor; dataplotter; dpm data processing machine; processing machine, data; processor数据处理率: processing data rate数据处理盘: data panel数据处理器: data handler; data peocessor; data proce or; data processing system; data processor数据处理区: dpa data processing area数据处理仪: digital treating meter数据处理站: dps data processing station数据处理组: dpg data processing group数据处理中心;数据处理中心: data processing centre并行数据处理: parallel data processing