Database Development
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
User Name:
Password:
Remember me
 
Go Back   Dev Articles Community ForumsDatabasesDatabase Development

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Display Modes
 
Unread Dev Articles Community Forums Sponsor:
  #1  
Old May 26th, 2009, 10:19 AM
nmctech nmctech is offline
Registered User
Dev Articles Newbie (0 - 499 posts)
 
Join Date: May 2009
Posts: 1 nmctech User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 12 m 32 sec
Reputation Power: 0
Read from database or read from file ? Which is faster?

I am trying to optimize so I wasn't sure which one would be a better option.

I'm building an app that will use a database whose number of records will go into the thousands (and will continue to grow very quickly).

A lot of that information on the database (especially the older records) will be static and never be edited or deleted. But there are good chances those records will be pulled up frequently by search queries.

So would it be a better bet to remove those older records from the database and save it into a text/csv/xml file and read it from there ? Or keep it in the database ?

I am thinking in terms of what causes greater I/O and CPU instructions .. the file I/O or the database reads. Even a small difference could be substantial because I am expecting thousands of users to be querying the database (and adding/editing records to it) at real-time.

Thanks in advance!

Reply With Quote
  #2  
Old May 27th, 2009, 08:57 AM
dykebert's Avatar
dykebert dykebert is offline
Contributing User
Dev Articles Newbie (0 - 499 posts)
 
Join Date: Apr 2008
Posts: 427 dykebert User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 Days 2 m 9 sec
Reputation Power: 2
My gut reaction is that the database will be faster retrieving the records becuase that is what databases are built to do.

However making joins accross multiple tables takes time which is why data warehouses are de-normalized. Putting your data into a text based file (csv, xml) generally requires you to de-normalize it as well.

Based on the situation you described of mass entry and then reporting, I would build a database that had:
normalized tables for entry to ensure consistancy, etc
de-normalized tables for storing the data for quick reporting
indexes on the fields/columns that will typically be used for filtering data
__________________
P.S. I am looking for work. <grin>.

Reply With Quote
Reply

Viewing: Dev Articles Community ForumsDatabasesDatabase Development > Read from database or read from file ? Which is faster?


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump




 Free IT White Papers!
 
Create the Optimal Architecture for your Critical Applications
Warburton's the largest independently owned bakery in the UK faced a number of difficult challenges in providing the most robust yet efficient IT infrastructure for their organization's success. IBM's services combined with their xSeries servers created the perfect platform for their SAP environment with sufficient flexibility, and did so in very time effective fashion.

Request Your Free Technology Downloads!
 
Five Best Practices for Deploying a Successful Service-Oriented Architecture
This white paper describes the benefits you can expect with SOA, and how IBM can help take your business there.

Request Your Free Technology Downloads!
 
Gartner Magic Quadrant for Application Delivery Controllers
Gartner summarizes its view on Application Delivery Controllers, evaluates strengths and weaknesses of solutions, and provides Magic Quadrant reporting for a quick comparison across all vendors. Learn from Gartner how you can benefit from an all-in-one device like Citrix NetScaler that delivers the highest levels of availability, performance and security.

Request Your Free Technology Downloads!
 
Knowledge is Power
What you don't know can hurt you, and is likely costing you money and increasing your security risks during an era of scarce resources. This white paper proposes six key strategies that enterprise security managers can use to improve their network defense posture.

Request Your Free Technology Downloads!
 
Rationalizing the Multi-Tool Environment
The rationalized multi-tool approach is flexible, scalable and cost effective. It provides the necessary input to the IT service management business processes. It preserves prior investments in monitoring tools, empowers technologists to select the best tools with which to do their jobs, and enhances effective response to incidents.

Request Your Free Technology Downloads!
 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 




© 2003-2010 by Developer Shed. All rights reserved. DS Cluster 10 Hosted by Hostway
For more Enterprise Application Development news, visit eWeek