Todos los ejemplos a continuación son para sistemas Linux, probado en Ubuntu 20.04 y se asume que aws-cli está instalado.
La documentación de LocalStack sugiere como primer método de instalación localstack-cli. Pero existen alternativas que incluyen Docker, Helm, que pueden funcionar en ambientes de CI y Cockpit, este último ofrece una experiencia de escritorio de acuerdo a la documentación.
python3 -m pip install --user localstack
Terminado el proceso de instalación puedes comprobar que puedas acceder al comando.
localstack --help
Si este no está disponible el PATH puedes ejecutar como módulo de python.
python3 -m localstack --help
LocalStack soporta una gama amplia de herramientas para la nube, en la documentación pudes ver la lista de herramientas oficialmente soportadas.
aws-cli es la herramienta de línea de comandos que permite interactuar con los servicios de AWS, awscli-local es un reemplazo directo que interactua con LocalStack. Desde luego podemos utilizar aws-cli aws --endpoint-url=http://localhost:4566 en lugar de awscli-local, pero yo lo encuentro más comodo.
python3 -m pip install awscli-local
Ahora configurar la herramienta.
aws configure --profile localstack
Escribe test como valor para AWS Access Key ID y AWS Secret Access Key.
Deja el default None como valore de Default region name y Default output format.
Ya tienes lo mínimo necesario para empezar a desarrollar en LocalStack. Por ahora una función lambda sencilla servirá, AWS Lambda es uno de los servicios que LocalStack provee.
Inicia LocalStack, al finalizar la inicialización debes poder acceder a la URL http://localhost:4566 en tu navegador web.
localstack start
Primero crea el rol de ejecución para la función lambda, IAM es otra de las características de AWS que se emulan en LocalStack.
awslocal iam create-role --role-name lambda-ex --assume-role-policy-document '{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'
Puedes escribir tu propia Lambda para la prueba, pero aquí hay un repositorio en Github con el código de ejemplo que utilizo en esta publicación.
Clona el rpositorio y crea el archivo function.zip que vamos a publicar en LocalStack.
git clone https://github.com/ecelis/localstack-posts.git
cd localstack-posts
zip function.zip index.js
Publica la función, el número de cuenta de AWS en el ARN debe ser siempre 000000000000 sin especificar región, como en el ejemplo.
awslocal lambda create-function --function-name my-function \
--zip-file fileb://function.zip --handler index.handler --runtime nodejs12.x \
--role arn:aws:iam::000000000000:role/lambda-ex
Terminado el proceso con éxito puedes listar las funciones publicadas en LocalStack.
awslocal lambda list-functions
Invoca la función para tu satisfacción.
awslocal lambda invoke --function-name my-function out.txt
El resultado debe ser un JSON similar al siguiente.
{
"StatusCode": 200,
"LogResult": "",
"ExecutedVersion": "$LATEST"
}
Usa cat out.txt para mirar el contenido de out.txt que debe contener algo similar a los sigiente.
"2022/05/26/[$LATEST]d715a5ed8eddb6f5f731d0ecc584ad8d"
LocalStack ofrece una versión Pro con más servicios soportados, por ejemplo Lambda Layers o StepFunctions entre otros.
]]>One of the first things I noticed was the lack of support for Docker and going full with Kubernetes and OpenShift, which means no real need for Docker. Currently almost all my work is run in Docker containers, therefore I was glad to know that Podman is a direct replacement for Docker.
The Pod Manager tool podman is a container engine, just like Docker,
but without the requirement of a service daemon running all the time. It
is supported by Red Hat and all the operating system relatives, such as
Fedora and CentOS. It shares all docker sub-commands plus a few
others. For more information read the podman’s manual
pages.
podman talks directly to the image registry, the container and image
storage, and with the Linux kernel through the runC container runtime
process (not a daemon). Podman stores its containers and images in
different places than Docker, each user has its own
.local/share/containers directory, which allows to several user to use
Podman concurrently. Podman an Docker images are compatible. Podamn
allows to work with Kubernetes.
Make sure to have the latest updates for your operating system first,
then install podman which is available by default in Fedora 31 and
later releases.
sudo dnf update -y
sudo dnf install podman -y
Remove by hand the images you don’t want to migrate, old images, dangling intermediate build images, test, you know, anything that’s a waste of storage space. And then iterate through the Docker images and pull them into Podman.
for img in $(docker images --format '{{.Repository}}:{{.Tag}}'); \
do
echo $img; podman pull docker-daemon:$img; \
done
Review your images with podman, just like you’ll do with docker.
podman images
Now you’re ready to get rid of Docker. Stop the daemon, uninstall the
package, delete directories that could use space and drop the docker
user group.
docker system prune -a --volumes
sudo systemctl stop docker
sudo systemctl disable docker
sudo dnf remove moby-engine
sudo rm -rf /var/lib/docker /var/run/docker
sudo rm -rf /etc/docker ~/.docker
sudo groupdel docker
If you’re running CentOS or actual Red Hat Enterprise Linux, the install steps will change a bit.
So far I haven’t had major issues using Podman, however there are some
bumps. For example, I use docker-compose in several projects for
development, I know there is something called podman-compose,
which I need to review.
Some CI/CD pipelines in my projects expect the docker command, I
haven’t make my mind to change that, as I said images are compatible,
but I’d like to use the same tools all along the pipeline, I want to
avoid doing alias podman=docker.
Podman and Buildah for Docker user is a good read for starters.
]]>I decided to start a series of front end projects that I found at DEV.to “9 Projects you can do to become a frontend Master in 2020”. This post is about my own spin-off from the first project listed in that post by Simon Holdorf, a movie search app using React with hooks only, avoiding Class based components.
The fact that the challenge involves using the new hooks feature in React interests me particularly, because I recently revisited React after I learnt it one year and a half ago, then I left front end completely all this time. The base for my spin off comes from Samuel Omole’s tutorial at freecodecamp.org
The Header and Movie components are almost identical to the ones
Samuel wrote, but I took a few liberties in the Search and App
components. Also I prefer .jsx extensions for components and .js for
plain old javascript code. However I encourage you to read his tutorial
because he goes deeper in the logic behind the app and explains how the
hooks work, it is not my goal to repeat him.
What I do different?
create-react-app is the tool I am more
used to bootstrap React projects, honestly I haven’t tried many others
besides Gatsby. I became a patron at
CodeSandbox so why not use it for this
project? The codesandbox cli made a breeze to export the project I
had already bootstrapped from my laptop to the web editor. You can also
bootstrap your project directly in CodeSandbox and then export to a
Github repository and or publish it to Netlify. Go and check them out!
npm install -g codesandbox
codesandbox ./
Move App.js and App.css to components/App.js.
Add components/Header.jsx.
Also add a components/Movie.jsx file:
The next component components/Search.jsx will introduce hooks, the
new React feature that allow to handle state in functional components. I
really do not like the idea of classes in JavaScript and since I am not
a long time React developer I very much welcome hooks.
In the Search component the useState hook takes the initial state as
parameter and returns an array with the current state (similiar to
this.state but does not merge old and new states togheter) and a
function to update the state setSearchValue in this case, which is the
same as this.setState.
useEffect is a hook which allows to perform side effects in your
components, like fetching data from OMDB in this app. Other hook is
useReducer which works similar to redux reducers, it accepts state and
action and returns the current state and a dispatch method.
There are a couple of unbreakable rules for using hooks in React:
You can read more about Hooks in the React docs.
In summary, I will be using hooks over class based components in my current and next React projects, a good thing is that class and function components with hooks mix well, so no heavy refactoring of class components and slow adoption of hooks is a good approach for existing projects.
You can try and explore the code for the working movie app. Be ware! Since this is a demo the server-less function behind is not setup to scale and a heavy load of requests may take it down temporarily.
]]>Objects that can be used in for..of are called iterable.
Iterables must implement the method named Symbol.iterator. The result
of obj[Symbol.iterator] is called an iterator. An iterator must have
the method named next() that returns an object {done: Boolean, value:
any}, the value is the next value. The Symbol.iterator method is
called automatically by for..of, but we also can do it directly.
Array.from(obj[, mapFn, thisArg]) makes a real Array of an iterable or
array-like obj, and we can then use array methods on it. The optional
arguments mapFn and thisArg allow us to apply a function to each item.
Map an object or array to variables.
Object syntax:
let {prop : varName = default, ...} = object
Array syntax:
let [item1 = default, item2, ...rest] = array
The first item goes to item1; the second goes into item2, all the rest
makes the array rest.
Map, is a collection of keyed values, it allows objects to be keys and
provides a size property.
Set, is a collection of unique values, does not allow to reorder elements.
The following allow garbage-collection:
There ara also WeakMap and WeakSet which provide garbage collection,
I need to research more about these. Maybe the next post.
Grab yourself a copy of the mainline kernel repository
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
.config CONFIG_LOCALVERSION_AUTO=y
make localmodconfig
make silentoldconfig
make menuconfig
make -j4 all
make modules_install
make install
SRCTREE= linux
MYFILE= drivers/net/mydriver.c
cp $MYFILE $MYFILE.orig
vi $MYFILE # make your change
cd ..
diff -up $SRCTREE/$MYFILE{.orig,} > /tmp/patch
Multiple files
MYSRC= /dev/linux
tar xvzf linux-3.19.tar.gz
mv linux-3.19 linux-3.19-vanilla
diff -uprN linux-3.19-vanilla/Documentation/dontdiff \
linux-3.19-vanilla $MYSRC > /tmp/patch
Check your patches with the patch style checker scripts/checkpatch.pl
Numbers are double precision real numbers. If needed to specify an integer we should use the L suffix. There are a couple of special numbers, Inf and NaN which stand for Infinite and Not A Number.
Objects in R can have attributes
The c() function is used to create vectors.
x <- c(1, 3, 4, 7)
x <- vector("numeric", lenght=2)
When different objects are mixed in a vector, coercion occurs so that
every element in the vector is of the same class. Objects can be
explicitly coerced using the as.* function. If non-sense coercion is
tried the result is NA.
x <- 0:6
class(x)
as.numeric(x)
as.logical(x)
as.character(x)
Lists are very important in R, the element in a list are enclosed in double square brackets and can be accesed by their index in the list.
x <- list(1, "a", TRUE)
[[2]]
[2] "a"
Special vetor in R with a dimension attribute which is itself an integer vector of lenght 2 (nrow, ncol). Matrices are constructed column-wise.
m <- matrix(nrow = 2, ncol = 3)
dim(m)
attributes(m)
Matrices can also be created directly from vectors by adding a dimension attribute.
m <- 1:10
dim(m) <- c(2,5)
Binding can be used to create matrices also by using the functions
cbind() and rbind()
x <- 1:3
y <- 10:12
cbind(x,y)
rbind(x,y)
Are used to represent categorical data, can be ordered or unordered. One can think a factor as an integer vector where each integer has a label. Using factors with labels is better than using integers because factors al self-described.
The order of the labels can be set using the levels argument, this can be important for linear modeling.
x <- factor(c("yes", "yes", "no", "yes", "no"), levels = c("yes", "no"))
table(x)
unclass(x)
attr(,"levels")
Missing values are denoted by NA or NaN fr undefined mathematical operations. NA values can have a class also, so there are integer NA, character NA, etc. A NaN value is also an NA value, but not the opposite.
x <- c(1, 2, NA, 10, 3)
is.na(x)
is.nan(x)
Are used to store tabular data, they are represented as a special list has to have the same length. Each element can be thought of as a column and the length of each element of the list is the number of rows. Unlike matrices, data frames can store different classes of objects in each column. Data frames have a special attribute called row.names
x <- data.frame(foo = 1:4, bar = c(T, T, F, F))
x
nrow(x)
ncol(x)
R objects can have names, which is useful for writing readable code.
x <- 1:3
names(x)
x
names(x) <- c("foo", "bar", "norf")
x
x <- list(a = 1, b = 2, c = 3)
x
m <- matrix(1:4, nrow = 2, ncol = 2)
dimnames(m) <- list(c("a", "b"), c("c", "d"))
m
EOF
]]>There are two ways of currently running GNU Guix, either it could be installed on top of an existing GNU/Linux distribution, or installed directly on bare metal (or a virtual machine for the instance). Since the virtualization platform I use is VirtualBox, this was my primary target, in contrast qemu is the preferred choice of Guix developers, because its GPL license I assume.
Being in alpha state, the GNU Guix distribution provides a very basic USB image to boot and install the system, VirtualBox currently does not boot from USB, so I had to convert the installer raw image to some format which VirtualBox could handle. I tried VBoxManage at first, but it didn’t work or maybe I was unable to figure out how to do so, in the end I used qemu-img command to achieve the task.
In Gitorious you can find my notes and some guides which I wrote during this two days of the hackathon.
The latest GNU Guix installer image version available for download is
Guix-0.7 which is flawed, guix --version shows the right version but
the bundled packages are from Guix-0.6. It is not a showstopper, since
you can easily upgrade to the latest version right before installing to
your hard drive. This little bug slowed my install a few hours until I
found the solution when someone else hit the same problem in #guix at
freenode.net
The installer script is rather basic, do not expect anything fancy like hard disk re-partitioning, much less a GUI. It does not mean that it is hard to install, any mididly experienced unix user should be able to use fdisk, e2label, mount, sed and other basic commands to get a running GNU Guix system. One thing I missed though was the lack of vi or ed text editors, my keyboard is broken and keys ‘Z’ and left ‘CTRL’ don’t work. VirtualBox grabs hold of the right ‘CTRL’ for its own purposes and the only two editors available in the installer image are nano and zile (an Emacs clone), both unusable with my current keyboard.
Also the installer fetches pre-built binaries from hydra.gnu.org, sometimes this hosts is overloaded or simply unresponsive and you will get this “guix-system: error: build failed: unexpceted-end-of-file” message instead of the cool one “Installation finished. No error reported”, just re-run the last command until the task finishes successfully. If everything went well, now you should have a shiny new GNU Guix system ready to be booted into.
EOF
]]>Sadly I ain’t very fond of none of the available distros, and Slackware has an ARM port which is now officialy endorsed by Patrick Volkerdi himself. So I decided to give it a try.
SlackwareARM has official support for some boards and community support for some others, the main difference is that the official ones work with the standard Slackware installer and the ones supported by the community don’t. Luckly the SlackwareARM team provides a compressed mini-root filesystem with the bare minimal to bootstrap the system.
Here I’ll give only an overview of the steps needed to bootstrap SlackwareARM 14.1 on Cubieboard2. Detailed guidelines are available in Spanish language in Mi Kiwi.
All my hardware runs some sort of Unix like operating system, everything I describe here was done in a laptop running Slackware64 14.1 with multi-lib enabled.
u-boot-with-spl.bin to the mSDuImage kernel and install modules into
the rootfs directory.Cubieboard2.fex into the script.bin fileuEnv.txt fileboot.cmd into boot.scrrootfs/etc/fstabPlace the mSD card into the Cubieboard2 and boot it, if you have an USB
Serial converter you can watch/debug the boot and load of the kernel
attaching your terminal to the serial console using either, cu, minicom
or screen.
For the lazy bums out there, you can download either, a mSD card image file ready to boot, or the root filesystem compressed and the u-boot-with-spl.bin files to customize your install from MediaFire.
EOF
]]>Going straight to the point, directly under my control are around 10 cameras installed abroad the city where I currently live. This is not the first time I do security related development. And for the most part (thinking as an ex-volunteer firefighter) I really think that the current project which involves more than cameras is really helpful for the people in this city. Let’s temporarily forget the goal of the project.
I think about the cameras as the eyes of first response corps, certainly these days (almost) everyone carries a mobile phone capable of doing free calls to the emergency number, a big chunk of this mobile users own a so called smart-phone capable not only of making the emergency call, but also most of these gadgets are equipped with sensors and in some cases software capable of sending geo-location data and media to the emergency control centres. What this devices can’t provide neither replace is the training of first responders teams and dispatch operators.
So, the right mix of technology with trained and highly ethic people in charge of the cameras I was talking about in the first place, make the surveillance projects worth. The problem arises from an education and economics background. I can sleep fine every night, since I do a professional and transparent (to maximum allowed extent) work, I was grown educated by certain ethics and in a relatively comfortable environment. Speaking of my current work, all code I’ve written is being published with some sort of free or open source licence, so it can be openly audited. But what about the high rank officers, middle ground operators and other people with access to the technology and who are able to exploit it for their own purposes?
Technically, I or any person with the right skill set can create things that can be twisted into evil tools by other kind of skilled people, how do you control the later?
I could stop doing what I do, but some one else will do it anyways. The key is education which is the easiest part, since the other part, the economics are a bit more complicated to even think about, but easily controlled by education means though.
EOF
]]>One year or so passed and in July of 2004 I already felt very comfortable working in the shell, coding in Emacs and even playing my mp3 music collection and DVDs, things weren’t so smooth and fancy as today, there wasn’t transparent borders, or shadows, Ubuntu was in diapers and setting up the graphic desktop was a pain in the ass in your lucky day. But I learnt a lot about my hardware and the inner workings of the operating systems, not only Linux based ones.
I got an old IBM Pentium 100MHz with 48MB RAM and 20G Hard Drive along with an USRobotics 56k modem, the modem was intended to provide Internet connectivity for my main PC that ran Slackware, the internal modem of the later didn’t played well with Linux. Thinking about what to do with the Pentium machine I stumbled upon some on-line article about Hotmail. I thought of learning about servers by running my own web, email, ftp and any kind of server on the venerable IBM hardware, which proved to be a fine workhorse running by several months almost unatended with no downtime (actually the server ran several years, but I upgraded the hardware very soon), unless lack of power of the like. I wasn’t bound to dialup too much time, very soon in mid 2004 I leased a 256Kbps DSL connection which made perfect fit with the server project.
I started researching about pre-Microsoft, Hotmail’s original infrastructure, trying to emulate it if possible, it turned out that FreeBSD was the operating system used by Hotmail and qmail was the MTA. I already knew about the BSD family of unices, but never touched one of such beasts before. Again the task of learning was daunting at times, but very joyful overall. This time gained a lot of knowledge about the workings of the Internet, web servers and the way http requests/responses are handled (really helpful for my career as web apps developer), qmail and how the protocols SMTP, IMAP and POP work togheter, I used to host my own domain, both web and email and my own Subversion server. FreeBSD 4.1 was the operating system choice, also great for learning about true unix which is not the same as GNU/Linux.
In ten years I’ve seen the raise (meaning widespread use) of virtualization technologies which I started using in 2005 with Xen and VMware, later clouds came and I was also interested in learning about it, good old cfe.homeunix.org (which was the domain name of my server) then left the physical layer and went to heaven, well only to the cloud in the form of an AWS AMI.
And the learning path does not seem to end soon, today I got my first server on IPv6 (you need to be on IPv6 in order to be able to reach the website). Thanks to Hurricane Electric and their IPv6 Tunnelbroker service, I know a bit about IPv6 (I should know, I did the CCNA course) and the difference between v4 and v6 is more than 2 :-)
EOF
]]>