Sunday, November 6, 2011

Editing CSV Files













Before I start telling you about editing csv files, I would like to take a  moment to tell you about  a tool you may find useful.

 "Now, You Can Start Creating Your Very Own Minisite In Just A Few
Minutes
without Having To Hire An Expensive Web Designer!
Save Your Time
and Save Your Money Right Away!"

 Save time and money with a set of predefined templates. Learn more at Minisitebolt.

What are CSV Files?

CSV files are comma delimited files. They are used to move large amounts of data in tabular format, between applications, like a spreadsheet program such as Excel, and companies that are not connected.


In the past, most data was not stored online, and was only accessible on the computer the data was saved to, or the server that it was stored on.


Due to technological advances such as high speed, server side scripting languages like asp.net, and php, client side scripting languages like javascript, and the use of Ajax (Asynchronous JavaScript and XML)., more

applications are being designed for use on the Internet, via the browser of your choice.

Editing


Editing these files can be a challenge, if they are not saved in the right format. There are programs that can assist you; however, every program manipulates the data in a different way. You need to do your research well to find the right program for you and your data.


Computer programmers can build specialized programs to manipulate a csv, but for the common layman, csv file formats can be a challenge.


Client applications like spreadsheets, usually have the ability to save your data in a specified format like csv. Simply select, File - Save As, and choose the csv file format. This will allow you to easily save your spreadsheet file in a csv format.


Unlike the some of the commercial programs, like Microsoft Excel, your data may not be stored properly. I use an open source program called, Open Office 3.1. Unfortunately, the program stores my information with either an apostrophe or a quotation mark, and places them around each column, instead of between them.


Applications that can Assist in Editing CSV files


There are some applications that can assist you in editing csv file formats. One of thes programs is called
Ultra Edit by IDM Computer Solutions, at http://www.ultraedit.com/support/tutorials_power_tips/ultraedit/CSV_files.html.

Php Scripts


If you are a web developer, you should have access to the php scripting language.

I recently needed to manipulate a csv file to use it in a database. It was not in the correct format for me to use it.


The data was saved in the following format, "'column1' 'coloumn2'", I needed to reconfigure the file with a single

comma separating the columns.

Here is a sample of the text I was trying to manipulate:


AAA.TO' 'Allana Potash Corp'

AAB.TO'            'Aberdeen International Inc'
AAB.WT.TO'    'Aberdeen Intl Inc'
AAH.TO'            'Aastra Technologis'
AAV.TO'            'Advantage Oil & Gas Ltd'
AAV.DB.D.TO'    'Advantage Oil & Gas Ltd. 7.75% Debs'
AAV.DB.G.TO'    'Advantage Oil & Gas Ltd. 8.0% Debs'

To enable me to use it properly, I wrote a php script. It took me several hours, but I finally accomplished the task.


Since that time, I found out that it was all for nothing. I didn't spend allot of time going of the website that I downloaded it from, and could of saved allot of time and effort.


It is not perfect but is okay for my purposes. This code can serve as

the foundation for anyone who needs to manipulate csv files.

For anyone who is interested. Here is my code:


$text = file('tsx_list.txt'); //get file


foreach($text as $line) {


$trimmed1 = ltrim($line, "'");

$len_of_trimmed1 = strlen($trimmed1);
  //$trimmed3=(substr($trimmed1, 0, 12));
  //$trimmed3=(substr($trimmed1, 0, 7));
 
  if (substr($trimmed1, 6, 1) != "'")  //Will either be a apostrophe or a dot.
  {
    $part1=(substr
  }
  else{
    $part1=(substr($trimmed1, 0, 7)); //Store the first part of our new string.
  }
 
  //Check criteria to construct the second part of the new string.
 
  if (substr($trimmed1, 6, 1) != "'")  //Will either be a apostrophe or a dot.
  {
      $part2=(substr($trimmed1, 10,$len_of_trimmed1 - 1)); //Save long version
     
      $len_of_part2=strlen($part2);
      $part3=(substr($part2, 3, $len_of_part2)); //Refine part2 to take away first 2 characters.
      $trimmed2.=$part1;
      $trimmed2.=$part3;
      $trimmed2.="
";
      $trimmed2.="
";
  }
  else
  {
        $part2=(substr($trimmed1, 9, $len_of_trimmed1 - 1)); //Save short version
      $trimmed2.=$part1;
      $trimmed2.=$part2;
      $trimmed2.="
";
      $trimmed2.="
";
  }

}


echo $trimmed2;



?>




Conclusion


Editing CS Files can be a challenge, but doesn't need to be. When downloading data in csv format, you need to fully understand how the data is being formatted by the source. The source can be a website, or an application such as Excel, any client side application that can store a file in this format.


Programming for the Web