How to read a file in Golang?

This post will discuss how to read a file in Golang. We will use the below packages to work with the files.

• os package provides a platform-independent interface to perform the operating level operations.
• ioutil package provides easy-to-use utility functions to work with files without knowing much about the internal implementations.
• bufio package implements buffered IO, which helps us increase the performance and throughput of the input and output operations.
• log package implements a simple logging package. We will be using it throughout our programs. We would use the Fatal() function of the log package in our programs.

There are many ways to read a text file in Golang –

• We can read the whole file at once.
• or, Read a file line by line.
• or, Read a file in chunks.

Read the whole file at once

We can easily read the whole file at once and store it in a variable. But remember that we shouldn’t do it with large files. We would be using the ioutil.ReadFile() function to read the file and store the file’s content in a variable.

First, let’s store a file in the same directory where our program is. So, our folder structure will be something like the one below.

___
   |
   |_ _ _ _ readthisfile.txt
   |
   |_ _ _ _ codekru.go

Here, we will store some content in a readthisfile.txt file, and our program will go into the codekru.go file ( both of them are present in the same folder ).

readthisfile.txt file content

This is first line
This is second line
This is third line

codekru.go

package main

import (
	"fmt"
	"io/ioutil"
	"log"
)

func main() {

	fileContent, err := ioutil.ReadFile("readthisfile.txt") // reading the file

	if err != nil {
		log.Fatal("error occurred")
	}

	fmt.Println(string(fileContent)) // printing the contents of the file

}

Output after running the program –

This is first line
This is second line
This is third line

Now, what happened here?

But first, let’s discuss a bit about the ReadFile() function.

• Method declaration – func ReadFile(filename string) ([]byte, error).
• What does it do? It reads the file whose path is passed into the function’s arguments and returns the file’s content. It internally uses the os.ReadFile(fileName) method.
• What does it return? It returns the file’s content in a byte array along with the error. A successful call will return err == nil.

The internal implementation of the ReadFile function

func ReadFile(filename string) ([]byte, error) {
	return os.ReadFile(filename)
}

Now, let’s get back to our program.

• ioutil.ReadFile("readthisfile.txt") returned a byteArray and an error. We stored the byteArray in the “fileContent” variable, and the error was stored in the “err” variable.
• Then we placed an if condition such that if err value is not nil, we would log “error occurred” using the log.Fatal() function. Remember, the Fatal() function is equivalent to the Print() function, followed by a call to os.Exit(1).
• And lastly, we would be printing the file’s content using the string(fileContent). We can’t simply print the fileContent variable because it’s a byte array, and we would need it to be converted to a string. string(fileContent) converts the byte array into a string.

Read a file line by line

Let’s use the same file and program which we used earlier and try to read the file line by line.

codekru.go file

package main

import (
	"bufio"
	"fmt"
	"log"
	"os"
)

func main() {

	// opening the file using Open function
	file, err := os.Open("readthisfile.txt")

	// If there was an error in opening the file, then report it and exit
	if err != nil {
		log.Fatal(err)
	}

	// creating a scanner for the file
	scanner := bufio.NewScanner(file)

	// loop through the scanner until it returns false
	for scanner.Scan() {
		fmt.Println(scanner.Text())
	}

	// If there is an error in scanning the file, then report it and exit
	if scanner.Err() != nil {
		log.Fatal(scanner.Err())
	}

	// closing the file
	err = file.Close()

	// If there is an error in closing the file, then report it and exit
	if err != nil {
		log.Fatal(err)
	}

}

Output –

This is first line
This is second line
This is third line

So, what happened here?

• os.Open() opens the file passed in the arguments. If any error is encountered while opening the file, it will return the same. Otherwise, the error would be nil.
• Then we use the scanner to read the file and scan it line by line till the contents of the file end.
• And finally, we are closing the file.

Read a file in chunks

Reading the whole file at once seems to be an easy approach, but sometimes we need to make our program a bit optimized from a memory management point of view. Golang provides a way of reading a file in chunks instead of reading it whole or even line by line. Because reading line by line might also be inefficient if the size of a line is too big.

We will again be using the readthisfile.txt and the same project structure. We will just be changing our codekru.go file.

codekru.go file

package main

import (
	"fmt"
	"io"
	"log"
	"os"
)

func main() {

	// the number of bytes that we will be reading at a time
	const BufferSize = 10

	file, err := os.Open("readthisfile.txt")
	if err != nil {
		log.Fatal(err)
	}

	buffer := make([]byte, BufferSize)

	for {

		// reading a file upto buffer
		bytesread, err := file.Read(buffer)

		// break from the for loop if end of file happened
		if err == io.EOF {
			break
		}

		// printing the buffer
		fmt.Println("string read: ", string(buffer[0:bytesread]))
	}

	// closing the file
	err = file.Close()

	// If there is an error in closing the file, then report it and exit
	if err != nil {
		log.Fatal(err)
	}

}

Output –

string read:  This is fi
string read:  rst line

string read:  This is se
string read:  cond line
string read:
This is t
string read:  hird line

Each character in a string is one byte. So, you can even count if the program reads 10 bytes at a time or not.

We hope that you have liked the article. If you have any doubts or concerns, please feel free to write us in the comments or mail us at admin@codekru.com.