Wednesday, July 15, 2020

How to Install Docker on Linux Ubuntu

1- Update Ubuntu's local database:


sudo apt-get update

2- Download Docker's dependencies:


sudo apt-get install apt-transport-https ca-certificates curl software-properties-common -y

3- Add Docker's GPG key:


curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

4- Install Docker's repository:


sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu  $(lsb_release -cs)  stable"

5- Update Ubuntu's repositories:


sudo apt-get update

6- Install Docker:


sudo apt-get install docker-ce -y

7- Add your user to the docker group:


sudo usermod -aG docker $USER

8- Log out and log back in so that the changes made take effect:


newgrp docker


Friday, June 26, 2020

How to Install Apache Spark on Linux Ubuntu

First of all, you need to have Java already installed; if you don't I recommend this article to install Java. Then you will need to install Scala:

Once done installing Scala, you will need to download the Apache Spark binaries archive and install it:

Afterward and in order to avoid moving to Spark's bin directory every time you want to interact with it, add Spark's bin directory to the PATH system's variable. To do so open .bashrc script vim .bashrc and add the following lines in the bottom of the file:

That's it, you only have to apply the last modifications running source ~/.bashrc

Monday, June 22, 2020

How to Install MongoDB on Linux Ubuntu 18.04

1- Import the MongoDB public GPG Key:
2- Create the /etc/apt/sources.list.d/mongodb-org-4.2.list file for Ubuntu 18.04 (Bionic):
3- Reload the local package database:
4- Install MongoDB:
5- Start MongoDB service:
6- Start a Mongo shell:
8- Create users database:
9- Create an adminstrator user:
10- Exit the Mongo shell:
11- Open the MongoDB configuration file sudo vim /etc/mongod.conf and edit the following parameters:
12- Restart MongoDB process:

How to Install Apache Kafka on Linux Ubuntu

In order to install Apache Kafka you should have Java installed and JAVA_HOME environment variable set; if not follow the first instructions of this article.
1- Download Zookeeper:

2- Extract the archive content:

3- Create Zookeeper configuration file from the sample provided:

4- Open the startup script file vim ~/.bashrc and add the following content:

5- activate the new path settings:

6- Start zookeeper service:

7- Download Kafka:

8- Create an installation folder for Kafka:

9- Extract the archive content:

10- Open the startup script file vim ~/.bashrc and add the following content:

11- activate the new path settings:

12- Create a shortcut to Kafka configuration file:

13- Edit listners parameter within Kafka configuration file i.e.:
listeners=PLAINTEXT://thedevopsnerdworld.com:9092
14- Start Kafka service:

Saturday, June 20, 2020

The Best Way to Install Java JDK and Eclipse IDE on Linux Ubuntu Desktop

In order to install Java on your ubuntu, I assume you are working from home directory cd ~ or cd $HOME which in my case is "/home/omar" or use sudo su - to switch to root user and avoid typing sudo everytime you need root privilege. To start, open a new terminal Ctrl+Alt+T and run the following command: then run the command below to find JDK installation directory: Afterward, open the startup script file vim ~/.bashrc and add the following lines in the bottom: then run the following command to activate the new path settings immediately: Now that you have java installed, go to Eclipse IDE download page: https://www.eclipse.org/downloads/download.php?file=/oomph/epp/2020-06/R/eclipse-inst-linux64.tar.gz and download the Eclipse installer archive. Once downloaded, extract the archive content by running the command below: After extracting the installer, open vim ~/eclipse-installer/eclipse-inst.ini file and append the following configuration: then launch the installer: cd ~/eclipse-installer/ && ./eclipse-inst and choose Eclipse IDE for Enterprise Java Developers. Once you have Eclipse installed, open its configuration file vim ~/eclipse/jee-2020-06/eclipse/eclipse.ini and add the following: now, create a shortcut for eclipse by creating a new file vim ~/eclipse.desktop and inserting the following content into it: then install the shortcut: That's it, congratulation! from now on you can find your Eclipse among Ubuntu applications.

Bonus: 

 You can also install Lombok utility within eclipse; all you have to do is go to Lombok download page: https://projectlombok.org/download and download the jar. Once the jar downloaded, launch it by running java -jar lombok.jar and specify the eclipse.ini file location /home/omar/eclipse/jee-2020-06/eclipse/eclipse.ini

Tuesday, October 29, 2019

How To Create A Lambda Function On AWS With Python That Creates An SQS Every Morning And Deletes It Every Night

First of all, go to Lambda service page on AWS, then create a function with a role having AmazonSQSFullAccess policy:


Afterward, inject the following code into the function:

import boto3

sqs = boto3.client('sqs')

def lambda_handler(event, context):
    if 'morning' in event['resources'][0]:
        # Create queue if morning event
        sqs.create_queue(
            QueueName='testEnvQueue',
        )
        print('Creating queue')
    if 'night' in event['resources'][0]:
        # Delete queue if night event
        QUEUE_URL = sqs.get_queue_url(
            QueueName='testEnvQueue'
        )['QueueUrl']
        sqs.delete_queue(
            QueueUrl=QUEUE_URL
        )
        print('deleting queue')

Next, at the top of the page hit Add trigger button:


And select CloudWatch Events, then select Create a new rule. In the Rule name field, make sure your name contains 'morning' and in the Schedule expression field enter something like rate(2 minutes) which will create an SQS queue named 'testEnvQueue' every 2 minutes just for the purpose of testing:


 Finally, add another CloudWatch event to simulate night to delete the created queue.

Monday, October 28, 2019

Interacting with AWS SQS Buffer Using Python: Example

First of all, make sure you have AWS CLI installed on your local machine by checking the existence  of both files "~/.aws/config" and "~/.aws/credentials":

 ~/.aws/credentials:

[default]
aws_access_key_id = XXX
aws_secret_access_key = XXXXXX

~/.aws/config:

[default]
region = us-east-2
output = text

In case you did not find them, check out the links at the bottom of this article. Then, create an SQS queue by executing the following Python's script:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import boto3
sqs = boto3.client('sqs')
sqs.create_queue(QueueName='translation_requests.fifo',
                 Attributes={'FifoQueue': 'true'})

Afterward, you should find the SQS queue named 'translation_requests.fifo' listed on the AWS console as follows:


Now that your queue has been created, create a Python list containing a list of phrases inside a Python file named 'phrase_list.py' as follows:

phrases = [
    "Hello!",
    "How are you?",
    "What’s up?",
    "I’m fine. And you?",
    "Please.",
    "Thank you.",
    "Thank you very much.",
    "You’re Welcome",
    "Goodbye.",
    "See you soon.",
    "Cheers!",
    "Excuse me.",
    "I’m sorry.",
    "What’s your name?",
    "My name is",
    "Nice to meet you.",
    "Where are you from?",
    "I’m from",
    "I’d like to introduce my friend",
    "How old are you?",
    "I’m 33 years old.",
    "What do you do for a living?",
    "I’m a doctor",
    "What do you do for fun?",
    "What are your hobbies?",
    "Do you speak English?",
    "Could you please speak a little slower?",
    "Could you write that down?",
    "Could you repeat that?",
    "How much?",
    "Can I pay by credit card?",
    "Here you go.",
    "What time do you open?",
    "Do you have anything cheaper?",
    "It’s too expensive.",
    "Here’s my passport.",
    "Do I have to change trains?",
    "Do I need a seat reservation?",
    "Is this seat taken?",
    "Could you call me a taxi?",
    "Could you let me know when to get off?",
    "Could you recommend a good restaurant?",
    "What would you recommend?",
    "What are some local specialties?",
    "What is the special of the day?",
    "Could I see the menu, please?",
    "That was delicious!",
    "This isn’t what I ordered.",
    "Can I buy you a drink?",
    "Let’s have another!",
    "Where can I find tourist information?",
    "Do you have a map?",
    "Can you show me that on the map?",
    "What is the entrance fee?",
    "What is that building?",
    "What is there to see around here?",
    "I have a reservation.",
    "Could I see the room?",
    "I’d like to stay for five nights.",
    "Is breakfast included?",
    "Could I get a different room?",
    "Is there a restaurant here?",
    "Is there pharmacy nearby?",
    "Can I use your phone?"
]

Then, create a python script to push the previous list of phrases into your SQS queue as follows:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import boto3
import json
import random
import time
import uuid

from phrase_list import phrases

sqs = boto3.client('sqs')

QUEUE_URL = sqs.get_queue_url(QueueName='translation_requests.fifo'
                              )['QueueUrl']


def send_requests_to_sqs():
    message = {'phrase': random.choice(phrases),
               'lang_code': random.choice(['de', 'es', 'fr'])}

    json_message = json.dumps(message)
    sqs.send_message(QueueUrl=QUEUE_URL, MessageBody=json_message,
                     MessageGroupId='Translations',
                     MessageDeduplicationId=str(uuid.uuid4()))
    print('Message sent to SQS.')


i = 0
while i < 50:
    i = i + 1
    time.sleep(2)
    send_requests_to_sqs()

Once executed, check the number of available messages inside your queue on the AWS console:


Thereafter, write another Python script to pull messages from the queue and then translate them using the TranslateText API as follows:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import boto3
import json
import time

sqs = boto3.client('sqs')
translate = boto3.client('translate')

QUEUE_URL = sqs.get_queue_url(QueueName='translation_requests.fifo'
                              )['QueueUrl']


def consume_messages():

    # Load messages from the SQS queue

    response = sqs.receive_message(QueueUrl=QUEUE_URL,
                                   MaxNumberOfMessages=1)['Messages'][0]
    receipt_handle = response['ReceiptHandle']
    message_body = json.loads(response['Body'])

    # Translate the message to the language

    translation = translate.translate_text(Text=message_body['phrase'],
            SourceLanguageCode='en',
            TargetLanguageCode=message_body['lang_code'])
    print('Translated "' + message_body['phrase'] + '" to: "' \
        + translation['TranslatedText'] + '"')
    sqs.delete_message(QueueUrl=QUEUE_URL, ReceiptHandle=receipt_handle)


i = 0
while i < 1000:
    i += 1
    try:
        consume_messages()
        time.sleep(1)
    except:
        i = 1000
        print ('oops')

Now that you have consumed the messages stored in the queue, you can proceed to delete the queue by running the following Python script:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import boto3

sqs = boto3.client('sqs')

QUEUE_URL = sqs.get_queue_url(QueueName='translation_requests.fifo'
                              )['QueueUrl']

sqs.delete_queue(QueueUrl=QUEUE_URL)

--------------------------------------------------------------------------------------------------------------------------

You could have emptied the queue rather than deleted it by running the following code:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import boto3

sqs = boto3.client('sqs')

QUEUE_URL = sqs.get_queue_url(QueueName='translation_requests.fifo'
                              )['QueueUrl']

sqs.purge_queue(QueueUrl=QUEUE_URL)

--------------------------------------------------------------------------------------------------------------------------

Useful links:

1. Download and install the AWS CLI
2. Configure the AWS CLI
3. Install Boto3